Oct 30 22:18:03.647181 ip-10-0-128-97 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 22:18:04.130069 ip-10-0-128-97 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:18:04.130469 ip-10-0-128-97 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 22:18:04.130469 ip-10-0-128-97 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:18:04.130469 ip-10-0-128-97 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:18:04.133071 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133002    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133081    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133085    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133087    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133090    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133092    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133095    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133098    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133100    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133103    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133105    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133108    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133111    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133113    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133117    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:04.133114 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133120    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133123    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133125    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133128    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133130    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133133    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133135    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133138    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133140    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133143    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133145    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133147    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133150    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133152    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133154    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133157    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133159    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133162    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133164    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133167    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:04.133456 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133169    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133171    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133174    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133176    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133178    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133181    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133183    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133186    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133188    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133191    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133193    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133196    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133200    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133202    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133205    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133208    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133210    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133212    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133215    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133217    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:04.133894 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133220    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133222    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133224    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133227    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133229    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133231    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133234    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133238    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133242    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133245    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133248    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133251    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133254    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133256    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133260    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133263    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133265    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133267    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133270    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:04.134374 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133272    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133275    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133277    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133280    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133282    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133289    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133292    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133295    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133298    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133300    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133303    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133305    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133307    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133310    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133312    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133315    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133317    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133319    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133323    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:04.134813 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133327    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133330    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133332    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133334    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133337    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133339    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133341    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133345    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133347    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133356    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133359    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133362    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.133364    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134097    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134104    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134107    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134110    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134112    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134115    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:04.135259 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134119    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134122    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134124    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134127    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134131    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134134    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134137    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134139    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134142    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134144    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134147    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134149    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134152    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134154    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134157    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134159    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134162    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134164    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134166    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:04.135683 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134169    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134171    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134174    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134176    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134179    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134181    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134184    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134186    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134189    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134192    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134194    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134196    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134199    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134201    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134205    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134207    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134210    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134212    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134214    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134217    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:04.136146 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134219    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134221    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134224    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134226    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134228    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134230    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134233    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134235    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134237    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134240    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134242    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134244    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134247    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134249    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134252    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134254    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134256    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134259    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134261    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134266    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:04.136599 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134268    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134271    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134274    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134276    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134279    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134281    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134284    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134287    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134289    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134292    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134294    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134297    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134299    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134302    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134304    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134306    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134309    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134311    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134314    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134316    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:04.137074 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134318    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134321    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134323    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134326    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134329    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134331    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134334    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134336    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134339    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134341    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134344    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134347    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134351    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134355    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134357    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134360    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134362    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134365    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134368    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:04.137546 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134370    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134373    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134820    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134825    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134828    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134831    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134833    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134836    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134838    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134841    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134844    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134846    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134849    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134851    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134853    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134856    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134858    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134861    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134864    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:04.137976 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134866    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134869    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134871    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134874    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134876    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134880    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134883    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134885    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134888    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134890    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134892    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134895    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134897    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134900    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134903    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134905    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134908    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134910    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134912    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134915    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:04.138513 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134918    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134921    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134923    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134925    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134928    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134930    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134932    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134935    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134937    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134939    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134942    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134944    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134946    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134949    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134951    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134955    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134958    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134963    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134965    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:04.138986 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134968    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134970    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134973    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134975    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134978    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134980    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134982    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134985    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134988    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134990    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134994    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.134998    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135000    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135004    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135006    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135009    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135011    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135013    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135016    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:04.139429 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135018    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135021    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135023    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135026    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135028    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135031    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135034    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135036    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135038    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135041    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135044    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135048    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135050    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135053    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135070    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135072    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135075    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135077    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135080    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135082    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:04.139889 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135084    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135087    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135090    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135092    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135095    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135097    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135099    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135102    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135104    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135106    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.135109    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136097    2571 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136106    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136112    2571 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136117    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136121    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136123    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136127    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136131    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136134    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136137    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136140    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 22:18:04.140354 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136145    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136149    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136152    2571 flags.go:64] FLAG: --cgroup-root=""
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136155    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136158    2571 flags.go:64] FLAG: --client-ca-file=""
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136161    2571 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136164    2571 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136168    2571 flags.go:64] FLAG: --cluster-domain=""
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136170    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136173    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136176    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136179    2571 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136183    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136187    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136190    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136193    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136195    2571 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136198    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136201    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136204    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136207    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136211    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136213    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136216    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136219    2571 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 22:18:04.140834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136221    2571 flags.go:64] FLAG: --enable-server="true"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136224    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136228    2571 flags.go:64] FLAG: --event-burst="100"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136231    2571 flags.go:64] FLAG: --event-qps="50"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136234    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136237    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136239    2571 flags.go:64] FLAG: --eviction-hard=""
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136244    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136247    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136250    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136253    2571 flags.go:64] FLAG: --eviction-soft=""
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136256    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136258    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136261    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136264    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136267    2571 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136269    2571 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136272    2571 flags.go:64] FLAG: --feature-gates=""
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136278    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136282    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136285    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136288    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136291    2571 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136294    2571 flags.go:64] FLAG: --help="false"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136297    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.141504 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136300    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136303    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136306    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136309    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136312    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136315    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136317    2571 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136320    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136323    2571 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136325    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136328    2571 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136331    2571 flags.go:64] FLAG: --kube-reserved=""
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136334    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136336    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136340    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136342    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136345    2571 flags.go:64] FLAG: --lock-file=""
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136348    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136351    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136353    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136359    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136361    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136364    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 22:18:04.142098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136367    2571 flags.go:64] FLAG: --logging-format="text"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136369    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136372    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136375    2571 flags.go:64] FLAG: --manifest-url=""
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136378    2571 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136382    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136385    2571 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136389    2571 flags.go:64] FLAG: --max-pods="110"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136392    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136395    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136397    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136400    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136403    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136406    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136409    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136415    2571 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136418    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136420    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136423    2571 flags.go:64] FLAG: --pod-cidr=""
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136426    2571 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136428    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136431    2571 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136434    2571 flags.go:64] FLAG: --port="10250"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136437    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136440    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-021491f51466d18d0"
Oct 30 22:18:04.142626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136443    2571 flags.go:64] FLAG: --qos-reserved=""
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136446    2571 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136449    2571 flags.go:64] FLAG: --register-node="true"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136451    2571 flags.go:64] FLAG: --register-with-taints=""
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136455    2571 flags.go:64] FLAG: --registry-burst="10"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136457    2571 flags.go:64] FLAG: --registry-qps="5"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136460    2571 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136463    2571 flags.go:64] FLAG: --reserved-memory=""
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136466    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136469    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136472    2571 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136475    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136477    2571 flags.go:64] FLAG: --runonce="false"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136480    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136483    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136486    2571 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136489    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136491    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136494    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136497    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136500    2571 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136502    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136505    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136508    2571 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136510    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136513    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 22:18:04.143225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136516    2571 flags.go:64] FLAG: --system-cgroups=""
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136518    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136523    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136527    2571 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136530    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136533    2571 flags.go:64] FLAG: --tls-min-version=""
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136536    2571 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136538    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136541    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136544    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136547    2571 flags.go:64] FLAG: --v="2"
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136551    2571 flags.go:64] FLAG: --version="false"
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136554    2571 flags.go:64] FLAG: --vmodule=""
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136558    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.136561    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136702    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136706    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136709    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136712    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136715    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136718    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136721    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136724    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:04.143811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136726    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136728    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136731    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136733    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136736    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136738    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136741    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136743    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136745    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136748    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136751    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136753    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136757    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136761    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136764    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136766    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136769    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136771    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136773    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:04.144389 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136776    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136778    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136780    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136783    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136785    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136788    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136790    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136792    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136795    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136797    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136800    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136802    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136805    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136808    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136810    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136813    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136815    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136818    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136820    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136823    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:04.144816 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136825    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136827    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136830    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136832    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136835    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136838    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136842    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136845    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136847    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136850    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136852    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136854    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136857    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136859    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136862    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136864    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136867    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136869    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136872    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136874    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:04.145288 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136876    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136879    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136881    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136883    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136886    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136889    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136891    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136894    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136896    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136900    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136903    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136905    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136908    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136910    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136912    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136915    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136919    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136925    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136929    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:04.145752 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136933    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136935    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136938    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136940    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136943    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136945    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136948    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136950    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136953    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136956    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136958    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136961    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136963    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136965    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136968    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136970    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136972    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136975    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136977    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:04.146201 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.136980    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.137005    2571 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137039    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137042    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137045    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137048    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137050    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137053    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137069    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137072    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137075    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137078    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137083    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137086    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137089    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137091    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:04.146639 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137094    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137096    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137099    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137101    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137104    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137106    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137109    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137112    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137114    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137117    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137119    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137121    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137124    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137127    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137129    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137132    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137134    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137137    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137140    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:04.147072 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137142    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137145    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137147    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137150    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137152    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137154    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137157    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137159    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137162    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137167    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137170    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137172    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137175    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137177    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137180    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137182    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137184    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137187    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137189    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:04.147774 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137192    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137194    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137197    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137199    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137202    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137204    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137206    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137209    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137211    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137214    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137216    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137219    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137222    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137224    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137227    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137229    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137232    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137234    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137237    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137239    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:04.148348 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137241    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137244    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137248    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137251    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137253    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137256    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137258    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137260    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137263    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137265    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137268    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137270    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137273    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137275    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137278    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137280    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137283    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137285    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137288    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137290    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:04.148843 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137293    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137295    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137298    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137300    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137302    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137305    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137309    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137312    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137315    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137318    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137320    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137323    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137326    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.137330    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.143638    2571 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.143653    2571 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.143668    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.143672    2571 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143755    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:04.149339 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143761    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143764    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143767    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143769    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143771    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143774    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143777    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143779    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143782    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143784    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143786    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143789    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143791    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143794    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143796    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143799    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143801    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143804    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143806    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:04.149771 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143808    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143811    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143813    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143816    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143818    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143821    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143823    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143826    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143829    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143837    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143840    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143843    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143845    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143848    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143851    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143855    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143858    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143861    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143864    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143866    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:04.150260 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143868    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143871    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143873    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143876    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143878    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143880    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143883    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143885    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143887    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143890    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143892    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143894    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143897    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143900    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143903    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143905    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143908    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143910    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143913    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:04.150705 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143915    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143918    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143920    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143928    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143931    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143933    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143935    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143938    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143940    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143943    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143945    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143947    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143950    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143952    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143955    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143957    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143959    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143962    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143964    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143966    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:04.151443 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143969    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143971    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143973    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143976    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143978    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143980    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143983    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143986    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143988    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143991    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143993    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143996    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.143998    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144001    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144003    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144006    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144013    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144016    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144018    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:04.151910 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144022    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144026    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144029    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144032    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144035    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144037    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144040    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144042    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.144086    2571 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144288    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144293    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144296    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144298    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144301    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144303    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144305    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:04.152386 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144308    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144310    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144312    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144315    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144318    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144320    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144322    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144325    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144327    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144329    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144332    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144335    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144337    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144340    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144347    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144350    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144353    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144355    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144357    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144360    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:04.152783 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144362    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144364    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144367    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144370    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144374    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144377    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144379    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144382    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144384    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144387    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144389    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144391    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144394    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144397    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144399    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144402    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144404    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144407    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144409    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:04.153365 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144412    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144414    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144416    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144419    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144421    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144423    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144426    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144428    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144436    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144439    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144443    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144446    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144449    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144452    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144454    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144456    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144458    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144461    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144463    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:04.153907 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144465    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144467    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144470    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144472    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144474    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144477    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144479    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144482    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144484    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144486    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144488    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144491    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144493    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144495    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144498    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144500    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144503    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144505    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144507    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:04.154434 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144510    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144512    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144514    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144522    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144525    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144527    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144530    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144532    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144534    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144537    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144539    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144541    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144544    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144546    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144549    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144551    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144553    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144555    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144558    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:04.154955 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144560    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144563    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144565    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.144587    2571 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144632    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144636    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144638    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144641    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144643    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144646    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144648    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144651    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144653    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144655    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144657    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:04.155418 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144660    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144662    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144670    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144672    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144675    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144677    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144679    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144682    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144684    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144686    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144688    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144691    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144693    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144696    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144698    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144701    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144703    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144706    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144709    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144711    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:04.155762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144713    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144716    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144718    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144720    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144723    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144725    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144727    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144730    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144732    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144735    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144737    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144739    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144741    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144744    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144746    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144753    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144756    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144758    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144761    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144763    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:04.156254 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144765    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144768    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144770    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144772    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144774    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144777    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144780    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144782    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144784    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144787    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144789    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144794    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144796    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144798    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144801    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144803    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144805    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144809    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144812    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:04.156836 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144815    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144818    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144821    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144824    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144827    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144829    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144832    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144834    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144836    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144844    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144847    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144849    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144851    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144854    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144856    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144858    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144861    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144863    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144865    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144868    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:04.157605 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144870    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144872    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144875    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144877    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144882    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144886    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144888    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144891    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144894    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144897    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144899    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144901    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144903    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144906    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144908    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:04.144910    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.144966    2571 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.146830    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 22:18:04.158135 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.148428    2571 server.go:1014] "Starting client certificate rotation"
Oct 30 22:18:04.158587 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.148500    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 22:18:04.158587 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.149227    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 22:18:04.158587 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.152627    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:18:04.158587 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.153752    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:18:04.162171 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.162157    2571 log.go:25] "Validated CRI v1 runtime API"
Oct 30 22:18:04.165037 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.165020    2571 log.go:25] "Validated CRI v1 image API"
Oct 30 22:18:04.165945 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.165921    2571 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 22:18:04.170414 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.170393    2571 fs.go:135] Filesystem UUIDs: map[4d0947a5-0271-4ced-b80d-f445e80bdd86:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a97a27a4-4e49-4a88-a76a-719ff70c57c6:/dev/nvme0n1p4]
Oct 30 22:18:04.170497 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.170412    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 22:18:04.176240 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.176126    2571 manager.go:217] Machine: {Timestamp:2026-10-30 22:18:04.173916192 +0000 UTC m=+0.305999507 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200163 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25d1ddf396fd3b32c148615834826f SystemUUID:ec25d1dd-f396-fd3b-32c1-48615834826f BootID:4d1d830c-ccaa-403f-aaff-6b03e2f00d5e Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f3:9a:88:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f3:9a:88:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:76:2e:09:17:99:1b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 22:18:04.176240 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.176236    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 22:18:04.176389 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.176354    2571 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 22:18:04.179486 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.179451    2571 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 22:18:04.179650 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.179497    2571 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-97.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 22:18:04.179694 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.179653    2571 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 22:18:04.179694 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.179661    2571 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 22:18:04.179694 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.179684    2571 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:18:04.180416 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.180404    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:18:04.181070 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.181046    2571 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 22:18:04.183386 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.183370    2571 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 22:18:04.184816 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.184805    2571 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 22:18:04.184927 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.184919    2571 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 22:18:04.184963 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.184949    2571 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 22:18:04.184963 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.184958    2571 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 22:18:04.185025 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.184968    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 22:18:04.185025 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.184976    2571 kubelet.go:405] "Adding apiserver pod source"
Oct 30 22:18:04.185025 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.184984    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 22:18:04.185250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.185239    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:18:04.191949 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.191925    2571 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 22:18:04.196786 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.196773    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:18:04.197742 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.197729    2571 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 22:18:04.197788 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.197759    2571 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 22:18:04.197922 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.197910    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 22:18:04.197952 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.197931    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 22:18:04.197952 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.197938    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 22:18:04.197952 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.197943    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 22:18:04.197952 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.197949    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 22:18:04.198053 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.197955    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 22:18:04.198588 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.198580    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 22:18:04.198616 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.198590    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 22:18:04.198616 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.198597    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 22:18:04.198616 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.198604    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 22:18:04.198616 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.198612    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 22:18:04.198727 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.198620    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 22:18:04.200150 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.200135    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 22:18:04.200202 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.200153    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 22:18:04.203641 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.203622    2571 server.go:1263] "Started kubelet"
Oct 30 22:18:04.203755 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.203690    2571 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 22:18:04.203812 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.203740    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 22:18:04.203875 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.203821    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 22:18:04.204019 ip-10-0-128-97 systemd[1]: Started Kubernetes Kubelet.
Oct 30 22:18:04.205214 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.205000    2571 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 22:18:04.205610 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.205575    2571 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-97.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 22:18:04.205862 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.205831    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-97.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 22:18:04.205947 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.205931    2571 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 22:18:04.207005 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.206990    2571 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 22:18:04.208683 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.208649    2571 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 22:18:04.212105 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.212088    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 22:18:04.213116 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.213100    2571 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 22:18:04.213244 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.213205    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:04.213695 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.213678    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 22:18:04.213695 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.213680    2571 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 22:18:04.213818 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.213717    2571 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 22:18:04.213818 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.213778    2571 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 22:18:04.213818 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.213789    2571 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 22:18:04.214173 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.214159    2571 factory.go:55] Registering systemd factory
Oct 30 22:18:04.214206 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.214188    2571 factory.go:223] Registration of the systemd container factory successfully
Oct 30 22:18:04.214385 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.214373    2571 factory.go:153] Registering CRI-O factory
Oct 30 22:18:04.214452 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.214386    2571 factory.go:223] Registration of the crio container factory successfully
Oct 30 22:18:04.214452 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.214429    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 22:18:04.214452 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.214442    2571 factory.go:103] Registering Raw factory
Oct 30 22:18:04.214452 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.214452    2571 manager.go:1199] Started watching for new ooms in manager
Oct 30 22:18:04.215360 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.215341    2571 manager.go:319] Starting recovery of all containers
Oct 30 22:18:04.217643 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.217623    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wz9lk"
Oct 30 22:18:04.218370 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.218348    2571 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-97.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 22:18:04.219293 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.218266    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-97.us-east-2.compute.internal.18e36e99f89b9425  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-97.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-97.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 22:18:04.203586597 +0000 UTC m=+0.335669899,LastTimestamp:2026-10-30 22:18:04.203586597 +0000 UTC m=+0.335669899,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-97.us-east-2.compute.internal,}"
Oct 30 22:18:04.221183 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.221158    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 22:18:04.227294 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.227278    2571 manager.go:324] Recovery completed
Oct 30 22:18:04.228081 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.227608    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wz9lk"
Oct 30 22:18:04.231308 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.231294    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:04.233045 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.233027    2571 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 22:18:04.233366 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.233345    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:04.233434 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.233379    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:04.233434 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.233390    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:04.234008 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.233992    2571 cpu_manager.go:226] "Starting" policy="none"
Oct 30 22:18:04.234008 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.234008    2571 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 22:18:04.234127 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.234016    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:04.234127 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.234025    2571 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 22:18:04.234693 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.234676    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:04.234758 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.234709    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:04.234758 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.234723    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:04.236006 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.235993    2571 policy_none.go:50] "Start"
Oct 30 22:18:04.236076 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.236012    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 22:18:04.236076 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.236027    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 22:18:04.238013 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.238001    2571 policy_none.go:44] "Start"
Oct 30 22:18:04.274689 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.274564    2571 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 22:18:04.278847 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.278836    2571 manager.go:341] "Starting Device Plugin manager"
Oct 30 22:18:04.278889 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.278874    2571 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 22:18:04.278889 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.278882    2571 server.go:87] "Starting device plugin registration server"
Oct 30 22:18:04.279054 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.279044    2571 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 22:18:04.279107 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.279075    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 22:18:04.279192 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.279173    2571 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 22:18:04.279308 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.279245    2571 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 22:18:04.279308 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.279251    2571 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 22:18:04.279624 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.279605    2571 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 22:18:04.279687 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.279650    2571 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:04.379525 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.379511    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:04.381191 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.380217    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:04.381191 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.380242    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:04.381191 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.380253    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:04.381191 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.380280    2571 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.386824 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.386810    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 22:18:04.386824 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.386826    2571 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 22:18:04.386934 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.386840    2571 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 22:18:04.386934 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.386869    2571 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 22:18:04.389042 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.389028    2571 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.389141 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.389050    2571 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-97.us-east-2.compute.internal\": node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:04.391550 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.391537    2571 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:04.414641 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.414621    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:04.487324 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.487298    2571 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"]
Oct 30 22:18:04.487398 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.487359    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:04.488156 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.488139    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:04.488237 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.488164    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:04.488237 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.488173    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:04.490260 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.490247    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:04.490407 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.490392    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.490469 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.490424    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:04.490888 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.490873    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:04.490961 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.490898    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:04.490961 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.490910    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:04.490961 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.490912    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:04.490961 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.490933    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:04.490961 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.490946    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:04.493005 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.492989    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.493103 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.493016    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:04.493685 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.493669    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:04.493750 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.493702    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:04.493750 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.493716    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:04.514521 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.514503    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.514593 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.514523    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.514593 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.514539    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e09f6b9a488f34d09a2deb5cc4fd598a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"e09f6b9a488f34d09a2deb5cc4fd598a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.515584 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.515570    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:04.517557 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.517538    2571 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found" node="ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.519947 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.519933    2571 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found" node="ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.614804 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.614782    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.614881 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.614806    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.614881 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.614823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e09f6b9a488f34d09a2deb5cc4fd598a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"e09f6b9a488f34d09a2deb5cc4fd598a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.614881 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.614850    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e09f6b9a488f34d09a2deb5cc4fd598a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"e09f6b9a488f34d09a2deb5cc4fd598a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.614971 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.614876    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.614971 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.614879    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.615864 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.615846    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:04.716976 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.716929    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:04.817344 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.817321    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:04.820470 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.820459    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.823206 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:04.823190    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:04.917620 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:04.917601    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:05.018023 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:05.017983    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:05.118380 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:05.118362    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:05.148513 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.148496    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 22:18:05.148808 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.148582    2571 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 22:18:05.218833 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:05.218816    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:05.231979 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.231947    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:13:05 +0000 UTC" deadline="2028-05-15 04:58:16.98577894 +0000 UTC"
Oct 30 22:18:05.231979 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.231974    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13494h40m11.753806893s"
Oct 30 22:18:05.319580 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:05.319556    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:05.420016 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:05.419982    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:18:05.437840 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:05.437815    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eb0646618f7571669a1ed8b2da1b80f.slice/crio-1a879aa3adc402f5517ef9cc23434f9de1f238b5ee9e9d36d908604df3c0665e WatchSource:0}: Error finding container 1a879aa3adc402f5517ef9cc23434f9de1f238b5ee9e9d36d908604df3c0665e: Status 404 returned error can't find the container with id 1a879aa3adc402f5517ef9cc23434f9de1f238b5ee9e9d36d908604df3c0665e
Oct 30 22:18:05.438151 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:05.438120    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09f6b9a488f34d09a2deb5cc4fd598a.slice/crio-6feaa293f879d3744c71c3a5cd4fbabb405776faf2c1b3a13198227247b500ad WatchSource:0}: Error finding container 6feaa293f879d3744c71c3a5cd4fbabb405776faf2c1b3a13198227247b500ad: Status 404 returned error can't find the container with id 6feaa293f879d3744c71c3a5cd4fbabb405776faf2c1b3a13198227247b500ad
Oct 30 22:18:05.442568 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.442552    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:18:05.512246 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.512226    2571 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:05.514324 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.514301    2571 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:05.526450 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.526431    2571 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:18:05.527953 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.527939    2571 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:18:05.528967 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.528954    2571 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:05.538788 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.538763    2571 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:18:05.552027 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:05.552007    2571 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:06.186038 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.186008    2571 apiserver.go:52] "Watching apiserver"
Oct 30 22:18:06.196512 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.196483    2571 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 22:18:06.197206 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.197182    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-kl9nq","openshift-multus/network-metrics-daemon-4pvll","kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal","openshift-dns/node-resolver-wqsgd","openshift-network-diagnostics/network-check-target-gtwvr","openshift-network-operator/iptables-alerter-5f2j9","openshift-ovn-kubernetes/ovnkube-node-82mjm","kube-system/global-pull-secret-syncer-hgsqv","kube-system/konnectivity-agent-cmh6h","openshift-image-registry/node-ca-dkmz4","openshift-multus/multus-4gvnm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf","openshift-cluster-node-tuning-operator/tuned-7wqr7"]
Oct 30 22:18:06.199713 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.199688    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.202511 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.202493    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.204850 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.204832    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jm6gb\""
Oct 30 22:18:06.204965 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.204952    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 22:18:06.205021 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.204982    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 22:18:06.205082 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.205038    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 22:18:06.205156 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.205145    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 22:18:06.205221 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.205211    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 22:18:06.205426 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.205413    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 22:18:06.205505 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.205493    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.206203 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.206186    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-szxgd\""
Oct 30 22:18:06.206258 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.206247    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 22:18:06.206341 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.206329    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 22:18:06.206432 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.206420    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 22:18:06.207117 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.207096    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:06.208710 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.208689    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:18:06.209177 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.209158    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qwcf4\""
Oct 30 22:18:06.209373 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.209343    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:06.209438 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:06.209411    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:06.209700 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.209684    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 22:18:06.210717 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.210454    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 22:18:06.210717 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.210590    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 22:18:06.210717 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.210652    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-glgqx\""
Oct 30 22:18:06.212329 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.212308    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:06.212563 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.212499    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 22:18:06.214797 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.214778    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:06.216315 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.216299    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:18:06.217316 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.217295    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:06.219708 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.219690    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:06.219789 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:06.219760    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:06.221962 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.221941    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:06.224425 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224406    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224711    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-host\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224779    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph67d\" (UniqueName: \"kubernetes.io/projected/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kube-api-access-ph67d\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224814    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-agent-certs\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224843    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-netns\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224866    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-log-socket\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224892    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cb74\" (UniqueName: \"kubernetes.io/projected/fb7545c5-92b9-4107-93d0-f0617277df0f-kube-api-access-4cb74\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224913    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-kubernetes\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224934    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-tmp\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224956    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.224979    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-socket-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225080    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-kubelet\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225127    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-slash\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225170    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-systemd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225194    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225250    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb7545c5-92b9-4107-93d0-f0617277df0f-ovn-node-metrics-cert\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.225309 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225290    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-tuned\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225321    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-netd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225346    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-modprobe-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225380    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-run\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225402    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d967b5ff-61dc-4c9a-8705-d735e0075db5-host-slash\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225426    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-var-lib-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225448    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-env-overrides\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225488    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysconfig\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225509    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-conf\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225531    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-systemd\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225563    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-var-lib-kubelet\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225589    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-668ht\" (UniqueName: \"kubernetes.io/projected/21192d3f-1a3f-4a32-ab4a-70fbcada9424-kube-api-access-668ht\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225615    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-hosts-file\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225641    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-node-log\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225707    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225730    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-tmp-dir\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:06.226092 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225753    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmhgd\" (UniqueName: \"kubernetes.io/projected/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-kube-api-access-hmhgd\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225775    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d967b5ff-61dc-4c9a-8705-d735e0075db5-iptables-alerter-script\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225798    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225821    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-device-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225844    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-etc-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225866    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-bin\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225889    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-config\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-sys\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225930    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-registration-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225953    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-etc-selinux\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.225975    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-systemd-units\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.226003    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.226116    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-script-lib\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.226141    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-lib-modules\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.226167    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nltc9\" (UniqueName: \"kubernetes.io/projected/d967b5ff-61dc-4c9a-8705-d735e0075db5-kube-api-access-nltc9\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.226190    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-sys-fs\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.226805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.226232    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-konnectivity-ca\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:06.227576 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.226257    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-ovn\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.227576 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.226280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.227576 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.226731    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:06.227576 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:06.226789    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:06.227902 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.227878    2571 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 22:18:06.247415 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.247389    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-s9lhf"
Oct 30 22:18:06.259597 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.259286    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-s9lhf"
Oct 30 22:18:06.305127 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.305101    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kkwnr\""
Oct 30 22:18:06.305325 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.305308    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jb8sb\""
Oct 30 22:18:06.305325 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.305317    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tp94j\""
Oct 30 22:18:06.305843 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.305824    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-4x7f7\""
Oct 30 22:18:06.306154 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306008    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jmj7l\""
Oct 30 22:18:06.306154 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306079    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 22:18:06.306154 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306086    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 22:18:06.306154 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306078    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 22:18:06.306154 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306138    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 22:18:06.306154 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306153    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 22:18:06.306457 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306292    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 22:18:06.306457 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306355    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 22:18:06.306457 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306410    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 22:18:06.306562 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306473    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 22:18:06.306562 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306501    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 22:18:06.306562 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306551    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 22:18:06.306664 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306577    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 22:18:06.306664 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.306605    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 22:18:06.315323 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.315302    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 22:18:06.327205 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327175    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-systemd-units\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327308 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327214    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327308 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327239    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-script-lib\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327308 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327261    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-lib-modules\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.327308 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327277    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-systemd-units\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327308 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327285    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nltc9\" (UniqueName: \"kubernetes.io/projected/d967b5ff-61dc-4c9a-8705-d735e0075db5-kube-api-access-nltc9\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327312    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327336    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cni-binary-copy\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327358    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-ovn\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327375    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327389    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph67d\" (UniqueName: \"kubernetes.io/projected/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kube-api-access-ph67d\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327403    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-agent-certs\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327417    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-log-socket\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327430    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cb74\" (UniqueName: \"kubernetes.io/projected/fb7545c5-92b9-4107-93d0-f0617277df0f-kube-api-access-4cb74\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327443    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-tmp\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327459    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327474    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327486    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb7545c5-92b9-4107-93d0-f0617277df0f-ovn-node-metrics-cert\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327500    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-tuned\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327516    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xdjz\" (UniqueName: \"kubernetes.io/projected/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-kube-api-access-7xdjz\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327530    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-kubelet\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327544    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-daemon-config\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.327549 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327557    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm9gc\" (UniqueName: \"kubernetes.io/projected/84f3a1bb-9373-443e-b957-1bf6ccb03477-kube-api-access-gm9gc\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-netd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327588    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-modprobe-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327601    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-run\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327614    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d967b5ff-61dc-4c9a-8705-d735e0075db5-host-slash\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327627    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84f3a1bb-9373-443e-b957-1bf6ccb03477-serviceca\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327641    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-var-lib-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-systemd\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327667    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-hosts-file\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327681    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhgd\" (UniqueName: \"kubernetes.io/projected/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-kube-api-access-hmhgd\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327698    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d967b5ff-61dc-4c9a-8705-d735e0075db5-iptables-alerter-script\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327712    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327736    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-konnectivity-ca\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327752    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-system-cni-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327766    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-etc-selinux\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327787    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327801    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-system-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.328250 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327816    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cnibin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327830    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-kubelet-config\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327845    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-bin\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327862    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-k8s-cni-cncf-io\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327877    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-conf-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327890    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-hostroot\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327905    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327943    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-host\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327946    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-script-lib\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327959    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npk9p\" (UniqueName: \"kubernetes.io/projected/5f9fcff5-40fb-422e-9319-50a1c4156131-kube-api-access-npk9p\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327975    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-socket-dir-parent\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.327991    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64j65\" (UniqueName: \"kubernetes.io/projected/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-kube-api-access-64j65\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328006    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-netns\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328023    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-kubernetes\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328040    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-socket-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-kubelet\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328104    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-slash\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.328747 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328127    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-systemd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328141    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-cnibin\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328155    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-lib-modules\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328157    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-binary-copy\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328227    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-ovn\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328240    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-var-lib-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328286    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-log-socket\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328291    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-systemd\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328006    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328733    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d967b5ff-61dc-4c9a-8705-d735e0075db5-host-slash\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328754    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-kubelet\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328799    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328828    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-modprobe-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328836    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-run\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328869    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-netns\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328884    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-hosts-file\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328902    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-slash\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.329534 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328923    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-kubernetes\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328925    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328942    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-host\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328960    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-etc-selinux\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328971    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-systemd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328981    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329009    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-bin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329032    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-multus\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329077    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-netns\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329101    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-multus-certs\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.328762    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-netd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329229    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-bin\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329238    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329269    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-env-overrides\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329303    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysconfig\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329339    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-conf\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329401    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-var-lib-kubelet\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330289 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329436    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-668ht\" (UniqueName: \"kubernetes.io/projected/21192d3f-1a3f-4a32-ab4a-70fbcada9424-kube-api-access-668ht\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329463    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-tmp-dir\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329474    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-conf\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329489    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-sys-fs\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329567    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysconfig\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329569    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d967b5ff-61dc-4c9a-8705-d735e0075db5-iptables-alerter-script\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329642    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329703    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-node-log\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329704    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-env-overrides\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329747    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329772    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-var-lib-kubelet\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329805    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-node-log\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329852    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-tmp-dir\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329879    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-sys-fs\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329774    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-device-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329943    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-device-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.330988 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.329975    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-etc-kubernetes\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330006    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-dbus\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330013    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330033    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-konnectivity-ca\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330037    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84f3a1bb-9373-443e-b957-1bf6ccb03477-host\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330090    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-etc-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330115    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-config\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330147    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-etc-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330191    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-sys\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330245    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-sys\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330253    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-registration-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330308    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-registration-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330477    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-config\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330573    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-socket-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:06.331559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.330572    2571 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 22:18:06.334182 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.334153    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-tuned\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.334333 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.334317    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-agent-certs\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:06.334397 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.334360    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-tmp\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:06.336206 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.336186    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb7545c5-92b9-4107-93d0-f0617277df0f-ovn-node-metrics-cert\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:06.342018 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:06.341963    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:06.342018 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:06.341988    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:06.342018 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:06.342001    2571 projected.go:196] Error preparing data for projected volume kube-api-access-xgg87 for pod openshift-network-diagnostics/network-check-target-gtwvr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:06.342209 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:06.342053    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87 podName:7cd8a3d9-fbbb-43c0-9920-345d2ab627c3 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:06.84203545 +0000 UTC m=+2.974118754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xgg87" (UniqueName: "kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87") pod "network-check-target-gtwvr" (UID: "7cd8a3d9-fbbb-43c0-9920-345d2ab627c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:07.566692 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.343686    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph67d\" (UniqueName: \"kubernetes.io/projected/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kube-api-access-ph67d\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:07.566692 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.343896    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cb74\" (UniqueName: \"kubernetes.io/projected/fb7545c5-92b9-4107-93d0-f0617277df0f-kube-api-access-4cb74\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:07.566692 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.345146    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-668ht\" (UniqueName: \"kubernetes.io/projected/21192d3f-1a3f-4a32-ab4a-70fbcada9424-kube-api-access-668ht\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:07.566692 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.345392    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nltc9\" (UniqueName: \"kubernetes.io/projected/d967b5ff-61dc-4c9a-8705-d735e0075db5-kube-api-access-nltc9\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:07.566692 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:06.345473    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmhgd\" (UniqueName: \"kubernetes.io/projected/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-kube-api-access-hmhgd\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:07.577496 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.577474    2571 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:07.610873 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.610813    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"6eb0646618f7571669a1ed8b2da1b80f","Type":"ContainerStarted","Data":"1a879aa3adc402f5517ef9cc23434f9de1f238b5ee9e9d36d908604df3c0665e"}
Oct 30 22:18:07.612267 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.612218    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"e09f6b9a488f34d09a2deb5cc4fd598a","Type":"ContainerStarted","Data":"6feaa293f879d3744c71c3a5cd4fbabb405776faf2c1b3a13198227247b500ad"}
Oct 30 22:18:07.649599 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649569    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm9gc\" (UniqueName: \"kubernetes.io/projected/84f3a1bb-9373-443e-b957-1bf6ccb03477-kube-api-access-gm9gc\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:07.649599 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649606    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84f3a1bb-9373-443e-b957-1bf6ccb03477-serviceca\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:07.649854 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649635    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-system-cni-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.649854 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649657    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.649854 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649706    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-system-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.649854 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649752    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cnibin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.649854 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649773    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-kubelet-config\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:07.649854 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649807    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-k8s-cni-cncf-io\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.649854 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649845    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-conf-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649867    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-hostroot\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649898    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649922    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npk9p\" (UniqueName: \"kubernetes.io/projected/5f9fcff5-40fb-422e-9319-50a1c4156131-kube-api-access-npk9p\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.649946    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-socket-dir-parent\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650016    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64j65\" (UniqueName: \"kubernetes.io/projected/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-kube-api-access-64j65\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650045    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-cnibin\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650074    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-binary-copy\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650096    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-bin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650150    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-multus\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650173    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-netns\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650195    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-multus-certs\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650277    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650305    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650327    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-etc-kubernetes\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650368    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-dbus\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650391    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84f3a1bb-9373-443e-b957-1bf6ccb03477-host\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650425    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650447    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cni-binary-copy\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650505    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjz\" (UniqueName: \"kubernetes.io/projected/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-kube-api-access-7xdjz\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650513    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-system-cni-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650595    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-kubelet\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.650701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650631    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-daemon-config\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.651058 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.650719    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84f3a1bb-9373-443e-b957-1bf6ccb03477-serviceca\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.651584    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-etc-kubernetes\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.651699    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.651780    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-multus\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.651895    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-dbus\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:07.652118    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:07.652199    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs podName:e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:08.152178702 +0000 UTC m=+3.065655547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs") pod "network-metrics-daemon-4pvll" (UID: "e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652252    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652260    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-daemon-config\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652308    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-cnibin\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652454    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84f3a1bb-9373-443e-b957-1bf6ccb03477-host\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652495    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-conf-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652531    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-system-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652580    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cnibin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652584    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-bin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652624    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-kubelet-config\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652654    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-k8s-cni-cncf-io\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652701    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-multus-certs\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.655344 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652777    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cni-binary-copy\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.656094 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:07.652811    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:07.656094 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652829    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-kubelet\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.656094 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:07.652857    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret podName:1ae3621a-b431-4675-8617-37461922d241 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:08.152839918 +0000 UTC m=+3.066316756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret") pod "global-pull-secret-syncer-hgsqv" (UID: "1ae3621a-b431-4675-8617-37461922d241") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:07.656094 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652859    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-netns\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.656094 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652914    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-socket-dir-parent\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.656094 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.652993    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.656094 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.653066    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.656094 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.653231    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-binary-copy\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.656094 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.653237    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-hostroot\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.664826 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.664792    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm9gc\" (UniqueName: \"kubernetes.io/projected/84f3a1bb-9373-443e-b957-1bf6ccb03477-kube-api-access-gm9gc\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:07.665278 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.665253    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64j65\" (UniqueName: \"kubernetes.io/projected/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-kube-api-access-64j65\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.665784 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.665759    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdjz\" (UniqueName: \"kubernetes.io/projected/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-kube-api-access-7xdjz\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:07.669131 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.669106    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npk9p\" (UniqueName: \"kubernetes.io/projected/5f9fcff5-40fb-422e-9319-50a1c4156131-kube-api-access-npk9p\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.754785 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.754749    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:07.757631 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.757604    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:18:07.760809 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.760786    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:18:07.762067 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.762044    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:18:07.775024 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.773174    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:18:07.775024 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:07.774947    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3ef0c70_9090_44d0_bb59_95ab1cf0c245.slice/crio-625923e6d4b4c44813dd2cdcf534575a058bc3fcf5722e0de8c33305b7394594 WatchSource:0}: Error finding container 625923e6d4b4c44813dd2cdcf534575a058bc3fcf5722e0de8c33305b7394594: Status 404 returned error can't find the container with id 625923e6d4b4c44813dd2cdcf534575a058bc3fcf5722e0de8c33305b7394594
Oct 30 22:18:07.777562 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:07.776227    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b6fa19a_6fb7_40b6_9073_0a52cfca6b79.slice/crio-3d27ea8736c54565ff75d36add2286e1cfaa9493c0724ae73dee3984bd75e9cd WatchSource:0}: Error finding container 3d27ea8736c54565ff75d36add2286e1cfaa9493c0724ae73dee3984bd75e9cd: Status 404 returned error can't find the container with id 3d27ea8736c54565ff75d36add2286e1cfaa9493c0724ae73dee3984bd75e9cd
Oct 30 22:18:07.781871 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.781849    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:07.784708 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:07.784654    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd967b5ff_61dc_4c9a_8705_d735e0075db5.slice/crio-893b03549c7592aedb81894711720581ed8582c228f7c92211071fcf98618961 WatchSource:0}: Error finding container 893b03549c7592aedb81894711720581ed8582c228f7c92211071fcf98618961: Status 404 returned error can't find the container with id 893b03549c7592aedb81894711720581ed8582c228f7c92211071fcf98618961
Oct 30 22:18:07.792527 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:07.792501    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0f05f12_85f1_4c1d_9115_e1cd2a336b55.slice/crio-836930fd1c99878552b3a1bbb4dc049f0c9bc41449fa23d5df4bb6c3b8b8d9bb WatchSource:0}: Error finding container 836930fd1c99878552b3a1bbb4dc049f0c9bc41449fa23d5df4bb6c3b8b8d9bb: Status 404 returned error can't find the container with id 836930fd1c99878552b3a1bbb4dc049f0c9bc41449fa23d5df4bb6c3b8b8d9bb
Oct 30 22:18:07.807084 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.807053    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:18:07.809003 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.808977    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4gvnm"
Oct 30 22:18:07.810834 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:07.810814    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:18:07.823640 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:07.823571    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f9fcff5_40fb_422e_9319_50a1c4156131.slice/crio-293cd367bcc86644f94637fd17e1da0102179470fbeb530dd5434e822ef5683b WatchSource:0}: Error finding container 293cd367bcc86644f94637fd17e1da0102179470fbeb530dd5434e822ef5683b: Status 404 returned error can't find the container with id 293cd367bcc86644f94637fd17e1da0102179470fbeb530dd5434e822ef5683b
Oct 30 22:18:07.827762 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:07.827739    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6f0136b_88a1_427c_92bc_5cc7ed9d7030.slice/crio-27165cb81007e61d7bd6e5e4707adcb2476998b09dac844ae734284963fe1c83 WatchSource:0}: Error finding container 27165cb81007e61d7bd6e5e4707adcb2476998b09dac844ae734284963fe1c83: Status 404 returned error can't find the container with id 27165cb81007e61d7bd6e5e4707adcb2476998b09dac844ae734284963fe1c83
Oct 30 22:18:08.154738 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.154698    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:08.154910 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.154777    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:08.154910 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.154827    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:08.155015 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:08.154963    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:08.155075 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:08.155028    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret podName:1ae3621a-b431-4675-8617-37461922d241 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:09.155010438 +0000 UTC m=+4.068487286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret") pod "global-pull-secret-syncer-hgsqv" (UID: "1ae3621a-b431-4675-8617-37461922d241") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:08.155445 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:08.155416    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:08.155592 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:08.155473    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs podName:e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:09.155457193 +0000 UTC m=+4.068934032 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs") pod "network-metrics-daemon-4pvll" (UID: "e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:08.155592 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:08.155553    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:08.155592 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:08.155568    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:08.155592 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:08.155580    2571 projected.go:196] Error preparing data for projected volume kube-api-access-xgg87 for pod openshift-network-diagnostics/network-check-target-gtwvr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:08.155803 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:08.155615    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87 podName:7cd8a3d9-fbbb-43c0-9920-345d2ab627c3 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:09.155604702 +0000 UTC m=+4.069081542 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgg87" (UniqueName: "kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87") pod "network-check-target-gtwvr" (UID: "7cd8a3d9-fbbb-43c0-9920-345d2ab627c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:08.481547 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.481349    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:13:07 +0000 UTC" deadline="2028-04-03 01:07:15.8890975 +0000 UTC"
Oct 30 22:18:08.482128 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.481549    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12482h49m7.407552774s"
Oct 30 22:18:08.606750 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.606616    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:08.607046 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:08.607010    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:08.621593 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.621561    2571 generic.go:356] "Generic (PLEG): container finished" podID="6eb0646618f7571669a1ed8b2da1b80f" containerID="0cfaa3a5781b78d2d096f38ae07c0a939c4933a312148f4255cf0c32d2755e08" exitCode=0
Oct 30 22:18:08.621717 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.621646    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"6eb0646618f7571669a1ed8b2da1b80f","Type":"ContainerDied","Data":"0cfaa3a5781b78d2d096f38ae07c0a939c4933a312148f4255cf0c32d2755e08"}
Oct 30 22:18:08.632171 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.632146    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dkmz4" event={"ID":"84f3a1bb-9373-443e-b957-1bf6ccb03477","Type":"ContainerStarted","Data":"6773a6575e687d07c3cfc972a450eb73e02320c5ef2c3d3b67ae40dd53805e9c"}
Oct 30 22:18:08.638554 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.638530    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5f2j9" event={"ID":"d967b5ff-61dc-4c9a-8705-d735e0075db5","Type":"ContainerStarted","Data":"893b03549c7592aedb81894711720581ed8582c228f7c92211071fcf98618961"}
Oct 30 22:18:08.647220 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.647196    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wqsgd" event={"ID":"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79","Type":"ContainerStarted","Data":"3d27ea8736c54565ff75d36add2286e1cfaa9493c0724ae73dee3984bd75e9cd"}
Oct 30 22:18:08.654148 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.654038    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" event={"ID":"fb7545c5-92b9-4107-93d0-f0617277df0f","Type":"ContainerStarted","Data":"933e5d368b5142c4f53b2b2438a6fe6ae1aae2936b0cfc2e599f5ea00e709727"}
Oct 30 22:18:08.656979 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.656927    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"e09f6b9a488f34d09a2deb5cc4fd598a","Type":"ContainerStarted","Data":"8fcd83ab29f9d728d838c745e96eec4f2489768936a15a1a7133d86f15b96f89"}
Oct 30 22:18:08.661463 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.661437    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4gvnm" event={"ID":"f6f0136b-88a1-427c-92bc-5cc7ed9d7030","Type":"ContainerStarted","Data":"27165cb81007e61d7bd6e5e4707adcb2476998b09dac844ae734284963fe1c83"}
Oct 30 22:18:08.663276 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.663251    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kl9nq" event={"ID":"5f9fcff5-40fb-422e-9319-50a1c4156131","Type":"ContainerStarted","Data":"293cd367bcc86644f94637fd17e1da0102179470fbeb530dd5434e822ef5683b"}
Oct 30 22:18:08.674360 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.673516    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal" podStartSLOduration=2.673501861 podStartE2EDuration="2.673501861s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:08.67329242 +0000 UTC m=+3.586769270" watchObservedRunningTime="2026-10-30 22:18:08.673501861 +0000 UTC m=+3.586978711"
Oct 30 22:18:08.690470 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.690401    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cmh6h" event={"ID":"f0f05f12-85f1-4c1d-9115-e1cd2a336b55","Type":"ContainerStarted","Data":"836930fd1c99878552b3a1bbb4dc049f0c9bc41449fa23d5df4bb6c3b8b8d9bb"}
Oct 30 22:18:08.694588 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.694509    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf" event={"ID":"f3ef0c70-9090-44d0-bb59-95ab1cf0c245","Type":"ContainerStarted","Data":"625923e6d4b4c44813dd2cdcf534575a058bc3fcf5722e0de8c33305b7394594"}
Oct 30 22:18:08.700458 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:08.700395    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7wqr7" event={"ID":"21192d3f-1a3f-4a32-ab4a-70fbcada9424","Type":"ContainerStarted","Data":"98caa419deb54a95e188a4123f92ee1f2606f0d73922c453841f54b8e17d44b5"}
Oct 30 22:18:09.162626 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:09.162551    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:09.162826 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:09.162622    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:09.162826 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:09.162715    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:09.162905 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.162851    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:09.162905 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.162903    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret podName:1ae3621a-b431-4675-8617-37461922d241 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:11.16288741 +0000 UTC m=+6.076364242 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret") pod "global-pull-secret-syncer-hgsqv" (UID: "1ae3621a-b431-4675-8617-37461922d241") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:09.163328 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.163307    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:09.163384 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.163366    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs podName:e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:11.163343008 +0000 UTC m=+6.076819839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs") pod "network-metrics-daemon-4pvll" (UID: "e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:09.163458 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.163444    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:09.163486 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.163464    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:09.163486 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.163475    2571 projected.go:196] Error preparing data for projected volume kube-api-access-xgg87 for pod openshift-network-diagnostics/network-check-target-gtwvr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:09.163544 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.163515    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87 podName:7cd8a3d9-fbbb-43c0-9920-345d2ab627c3 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:11.163503776 +0000 UTC m=+6.076980615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgg87" (UniqueName: "kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87") pod "network-check-target-gtwvr" (UID: "7cd8a3d9-fbbb-43c0-9920-345d2ab627c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:09.482314 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:09.482223    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:13:07 +0000 UTC" deadline="2028-07-25 18:32:43.465233696 +0000 UTC"
Oct 30 22:18:09.482314 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:09.482265    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15212h14m33.98297284s"
Oct 30 22:18:09.617448 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:09.616817    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:09.617448 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.617004    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:09.617813 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:09.617787    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:09.617957 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:09.617936    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:09.720603 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:09.720573    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"6eb0646618f7571669a1ed8b2da1b80f","Type":"ContainerStarted","Data":"39c7b37b69aaa2b310890904d2aa7a55501499f42b4a4e294e88f4328d375b14"}
Oct 30 22:18:10.606537 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:10.606502    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:10.607006 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:10.606633    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:11.179548 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.179461    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:11.179548 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.179513    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:11.179788 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.179563    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:11.179788 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.179755    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:11.179788 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.179771    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:11.179788 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.179780    2571 projected.go:196] Error preparing data for projected volume kube-api-access-xgg87 for pod openshift-network-diagnostics/network-check-target-gtwvr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:11.179991 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.179822    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87 podName:7cd8a3d9-fbbb-43c0-9920-345d2ab627c3 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:15.179809189 +0000 UTC m=+10.093286029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgg87" (UniqueName: "kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87") pod "network-check-target-gtwvr" (UID: "7cd8a3d9-fbbb-43c0-9920-345d2ab627c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:11.180190 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.180163    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:11.180286 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.180207    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret podName:1ae3621a-b431-4675-8617-37461922d241 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:15.180192261 +0000 UTC m=+10.093669088 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret") pod "global-pull-secret-syncer-hgsqv" (UID: "1ae3621a-b431-4675-8617-37461922d241") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:11.180286 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.180246    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:11.180286 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.180268    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs podName:e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:15.180260132 +0000 UTC m=+10.093736958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs") pod "network-metrics-daemon-4pvll" (UID: "e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:11.608736 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.606824    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:11.608736 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.606961    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:11.608736 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.608707    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:11.609242 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:11.608840    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:11.729052 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.728947    2571 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="564efb71befef180eb96f6e05e7c8c1f1e9e1b0d0a8cc5a9b72c60b4ad1e8c76" exitCode=0
Oct 30 22:18:11.729382 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.729316    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kl9nq" event={"ID":"5f9fcff5-40fb-422e-9319-50a1c4156131","Type":"ContainerDied","Data":"564efb71befef180eb96f6e05e7c8c1f1e9e1b0d0a8cc5a9b72c60b4ad1e8c76"}
Oct 30 22:18:11.739084 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.738559    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cmh6h" event={"ID":"f0f05f12-85f1-4c1d-9115-e1cd2a336b55","Type":"ContainerStarted","Data":"713ef5ddffa1820bb3a4e9370cf61b53cdcb7a5eff2e3c39a6816fd69ab4d3b3"}
Oct 30 22:18:11.757554 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.757499    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal" podStartSLOduration=5.757482481 podStartE2EDuration="5.757482481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:09.740164782 +0000 UTC m=+4.653641631" watchObservedRunningTime="2026-10-30 22:18:11.757482481 +0000 UTC m=+6.670959330"
Oct 30 22:18:11.811405 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.811326    2571 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:11.812322 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.812074    2571 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:11.832102 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:11.832038    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cmh6h" podStartSLOduration=3.167342911 podStartE2EDuration="6.832020132s" totalImagesPullingTime="3.664677221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:11.781342748 +0000 UTC m=+6.694819596" watchObservedRunningTime="2026-10-30 22:18:11.832020132 +0000 UTC m=+6.745496990"
Oct 30 22:18:12.605797 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:12.605761    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:12.605947 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:12.605888    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:13.607006 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:13.606977    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:13.607451 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:13.607117    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:13.607572 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:13.607554    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:13.607682 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:13.607647    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:13.746523 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:13.746443    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf" event={"ID":"f3ef0c70-9090-44d0-bb59-95ab1cf0c245","Type":"ContainerStarted","Data":"6dfe8fac749493e4bf9f7839d611aa997d862e22daf7c784f014fb71f30a7374"}
Oct 30 22:18:13.750904 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:13.749858    2571 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:18:13.750904 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:13.750827    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-dkmz4" event={"ID":"84f3a1bb-9373-443e-b957-1bf6ccb03477","Type":"ContainerStarted","Data":"e73ae95ad815b7091f2a0c669135031f34b0ea841c59142bbd2d007598279d14"}
Oct 30 22:18:13.768430 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:13.767979    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-dkmz4" podStartSLOduration=3.830220616 podStartE2EDuration="8.767965974s" totalImagesPullingTime="4.937745358s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:13.767462025 +0000 UTC m=+8.680938875" watchObservedRunningTime="2026-10-30 22:18:13.767965974 +0000 UTC m=+8.681442815"
Oct 30 22:18:14.606399 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:14.606367    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:14.606562 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:14.606455    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:14.914610 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:14.914570    2571 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.212693    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.212767    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.212817    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.212927    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.212986    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret podName:1ae3621a-b431-4675-8617-37461922d241 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:23.212967652 +0000 UTC m=+18.126444492 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret") pod "global-pull-secret-syncer-hgsqv" (UID: "1ae3621a-b431-4675-8617-37461922d241") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.213352    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.213389    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs podName:e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:23.213379627 +0000 UTC m=+18.126856454 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs") pod "network-metrics-daemon-4pvll" (UID: "e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.213440    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.213449    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.213457    2571 projected.go:196] Error preparing data for projected volume kube-api-access-xgg87 for pod openshift-network-diagnostics/network-check-target-gtwvr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:15.213511 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.213479    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87 podName:7cd8a3d9-fbbb-43c0-9920-345d2ab627c3 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:23.213472585 +0000 UTC m=+18.126949412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgg87" (UniqueName: "kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87") pod "network-check-target-gtwvr" (UID: "7cd8a3d9-fbbb-43c0-9920-345d2ab627c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:15.512801 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.512710    2571 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:18:14.914587038Z","UUID":"fc9d27de-85bc-490c-a967-7cac9c410aa6","Handler":null,"Name":"","Endpoint":""}
Oct 30 22:18:15.515579 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.514551    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 22:18:15.515579 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.514592    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 22:18:15.607901 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.607203    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:15.607901 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.607337    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:15.607901 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.607734    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:15.607901 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:15.607839    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:15.760291 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.760006    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-wqsgd" event={"ID":"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79","Type":"ContainerStarted","Data":"e2225f4d969cdee1dd5f66d3b63eed62d7b5b4e38c61aa34af66569b64e2e64a"}
Oct 30 22:18:15.763941 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.763910    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4gvnm" event={"ID":"f6f0136b-88a1-427c-92bc-5cc7ed9d7030","Type":"ContainerStarted","Data":"46f34f29237ee4ed0aaba4c9ba47c6a3c4fff9bd122865195d9cffc239e101a1"}
Oct 30 22:18:15.766951 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.766925    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf" event={"ID":"f3ef0c70-9090-44d0-bb59-95ab1cf0c245","Type":"ContainerStarted","Data":"8b3d081cb11caba39bd94da21736a24ff54baf499029ec9fcb6e5432240bab8b"}
Oct 30 22:18:15.778977 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.778739    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-wqsgd" podStartSLOduration=3.063395526 podStartE2EDuration="10.778723662s" totalImagesPullingTime="7.715328136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:15.778507965 +0000 UTC m=+10.691984820" watchObservedRunningTime="2026-10-30 22:18:15.778723662 +0000 UTC m=+10.692200512"
Oct 30 22:18:15.799284 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:15.799225    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-4gvnm" podStartSLOduration=3.496530791 podStartE2EDuration="10.79920926s" totalImagesPullingTime="7.302678469s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:15.798893106 +0000 UTC m=+10.712369960" watchObservedRunningTime="2026-10-30 22:18:15.79920926 +0000 UTC m=+10.712686123"
Oct 30 22:18:16.606219 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:16.606187    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:16.606687 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:16.606335    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:16.771710 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:16.771650    2571 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="ef572db646a056859c8a261de74d1b1378a5fbe30739449a19821d555d2177f9" exitCode=0
Oct 30 22:18:16.771878 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:16.771711    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kl9nq" event={"ID":"5f9fcff5-40fb-422e-9319-50a1c4156131","Type":"ContainerDied","Data":"ef572db646a056859c8a261de74d1b1378a5fbe30739449a19821d555d2177f9"}
Oct 30 22:18:16.774963 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:16.774937    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf" event={"ID":"f3ef0c70-9090-44d0-bb59-95ab1cf0c245","Type":"ContainerStarted","Data":"554eb2e2aef5fed21d49ef9ea4462cabcdd1f9c5b40fb5d79d821e46f59f7bcb"}
Oct 30 22:18:17.606193 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.606169    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:17.606305 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.606203    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:17.606305 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:17.606283    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:17.606945 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:17.606352    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:17.755003 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.754968    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:17.755161 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.755145    2571 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:18:17.755888 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.755864    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:18:17.774262 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.774220    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf" podStartSLOduration=4.913004871 podStartE2EDuration="12.774207261s" totalImagesPullingTime="7.86120239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:16.841202252 +0000 UTC m=+11.754679102" watchObservedRunningTime="2026-10-30 22:18:17.774207261 +0000 UTC m=+12.687684108"
Oct 30 22:18:17.779001 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.778965    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7wqr7" event={"ID":"21192d3f-1a3f-4a32-ab4a-70fbcada9424","Type":"ContainerStarted","Data":"277f4ad7f787452b45a7c2de4df90a5a5faf2a9e657407d57eae7ae4f3a98edc"}
Oct 30 22:18:17.780944 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.780901    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5f2j9" event={"ID":"d967b5ff-61dc-4c9a-8705-d735e0075db5","Type":"ContainerStarted","Data":"921aab5fb80fdb6b45f4dd2b6df5041b64450176b2e93d94aebaefe9ad59e8fe"}
Oct 30 22:18:17.803403 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.803300    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7wqr7" podStartSLOduration=3.040867865 podStartE2EDuration="12.803276347s" totalImagesPullingTime="9.762408482s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:17.803123317 +0000 UTC m=+12.716600179" watchObservedRunningTime="2026-10-30 22:18:17.803276347 +0000 UTC m=+12.716753194"
Oct 30 22:18:17.820614 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:17.820571    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5f2j9" podStartSLOduration=5.333769373 podStartE2EDuration="12.820555698s" totalImagesPullingTime="7.486786325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:17.819841808 +0000 UTC m=+12.733318658" watchObservedRunningTime="2026-10-30 22:18:17.820555698 +0000 UTC m=+12.734032546"
Oct 30 22:18:18.606404 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:18.606372    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:18.607020 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:18.606513    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:18.785638 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:18.785601    2571 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="1518d088ed8231415e441f620043a4e425b4e852f9a792b2508e60db1517c6d7" exitCode=0
Oct 30 22:18:18.785811 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:18.785708    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kl9nq" event={"ID":"5f9fcff5-40fb-422e-9319-50a1c4156131","Type":"ContainerDied","Data":"1518d088ed8231415e441f620043a4e425b4e852f9a792b2508e60db1517c6d7"}
Oct 30 22:18:19.605922 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:19.605698    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:19.606105 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:19.606072    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:19.606456 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:19.606436    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:19.606818 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:19.606537    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:19.791168 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:19.791076    2571 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="2fbfd55a6b2e83e764f5683041155809d3d5359d6023ed63cee555af4cd84dca" exitCode=0
Oct 30 22:18:19.791168 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:19.791114    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kl9nq" event={"ID":"5f9fcff5-40fb-422e-9319-50a1c4156131","Type":"ContainerDied","Data":"2fbfd55a6b2e83e764f5683041155809d3d5359d6023ed63cee555af4cd84dca"}
Oct 30 22:18:20.606083 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:20.606052    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:20.606276 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:20.606185    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:21.606265 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:21.606234    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:21.606265 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:21.606271    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:21.606640 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:21.606358    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:21.606640 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:21.606474    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:22.606745 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:22.606596    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:22.607166 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:22.606812    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:22.802292 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:22.802262    2571 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="78b671f692a8ebc3b97adcdd370a493cb80a749c224c2e181c4c680eaf085d86" exitCode=0
Oct 30 22:18:22.802393 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:22.802343    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kl9nq" event={"ID":"5f9fcff5-40fb-422e-9319-50a1c4156131","Type":"ContainerDied","Data":"78b671f692a8ebc3b97adcdd370a493cb80a749c224c2e181c4c680eaf085d86"}
Oct 30 22:18:22.805363 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:22.805337    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" event={"ID":"fb7545c5-92b9-4107-93d0-f0617277df0f","Type":"ContainerStarted","Data":"c78d486dfba4e76066523aa84fcdbfd8270a6efb3a5a91a4938d979cbb1db6f7"}
Oct 30 22:18:22.805434 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:22.805419    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" event={"ID":"fb7545c5-92b9-4107-93d0-f0617277df0f","Type":"ContainerStarted","Data":"4487ee613e7c3266f0d97996508d5961761f93167fad04dd163077e6e893e1b2"}
Oct 30 22:18:22.805487 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:22.805474    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" event={"ID":"fb7545c5-92b9-4107-93d0-f0617277df0f","Type":"ContainerStarted","Data":"d5baf9a6c7ad7107d48a68ebf92b27c663c055c91e2199fa925a1441291087f3"}
Oct 30 22:18:22.805537 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:22.805525    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" event={"ID":"fb7545c5-92b9-4107-93d0-f0617277df0f","Type":"ContainerStarted","Data":"0cf83b9acb0904a6a95b7423bf03c3276a469f0085ef823696d7008cf8097365"}
Oct 30 22:18:22.805590 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:22.805576    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" event={"ID":"fb7545c5-92b9-4107-93d0-f0617277df0f","Type":"ContainerStarted","Data":"18a5e82058f618478765d923fce5f3d3c140a6f6852e7473fbaa580330c176a7"}
Oct 30 22:18:22.805640 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:22.805628    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" event={"ID":"fb7545c5-92b9-4107-93d0-f0617277df0f","Type":"ContainerStarted","Data":"5f6eb300eb6dc42fef39240aadd457f145eedc4ba330819b710cf65c22d78fe9"}
Oct 30 22:18:23.272741 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:23.272695    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:23.272741 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:23.272732    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:23.272851 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:23.272751    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:23.272851 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.272826    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:23.272851 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.272831    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:23.272851 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.272844    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:23.272958 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.272860    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:23.272958 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.272864    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs podName:e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:39.272853906 +0000 UTC m=+34.186330733 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs") pod "network-metrics-daemon-4pvll" (UID: "e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:23.272958 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.272870    2571 projected.go:196] Error preparing data for projected volume kube-api-access-xgg87 for pod openshift-network-diagnostics/network-check-target-gtwvr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:23.272958 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.272886    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret podName:1ae3621a-b431-4675-8617-37461922d241 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:39.272870505 +0000 UTC m=+34.186347353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret") pod "global-pull-secret-syncer-hgsqv" (UID: "1ae3621a-b431-4675-8617-37461922d241") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:23.272958 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.272903    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87 podName:7cd8a3d9-fbbb-43c0-9920-345d2ab627c3 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:39.272893344 +0000 UTC m=+34.186370173 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgg87" (UniqueName: "kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87") pod "network-check-target-gtwvr" (UID: "7cd8a3d9-fbbb-43c0-9920-345d2ab627c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:23.605738 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:23.605721    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:23.605810 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:23.605740    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:23.605841 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.605806    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:23.605931 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:23.605914    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:23.809833 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:23.809813    2571 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="381e0671943f5d7a3d56ce16fea521894f49d70e6c993cc850b76aaf8c1f7d96" exitCode=0
Oct 30 22:18:23.810144 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:23.809846    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kl9nq" event={"ID":"5f9fcff5-40fb-422e-9319-50a1c4156131","Type":"ContainerDied","Data":"381e0671943f5d7a3d56ce16fea521894f49d70e6c993cc850b76aaf8c1f7d96"}
Oct 30 22:18:24.606158 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:24.606131    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:24.606266 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:24.606217    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:24.814584 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:24.814518    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" event={"ID":"fb7545c5-92b9-4107-93d0-f0617277df0f","Type":"ContainerStarted","Data":"a2311f46205806ee72ee98faab995332bd7e378651b7bf3df0ee2744cbb5df9e"}
Oct 30 22:18:24.817406 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:24.817384    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kl9nq" event={"ID":"5f9fcff5-40fb-422e-9319-50a1c4156131","Type":"ContainerStarted","Data":"45c0d077b8d7d81e92d177652ca9623d5d1899a5c33e95796b46890017cf61e6"}
Oct 30 22:18:24.847621 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:24.847585    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kl9nq" podStartSLOduration=8.972791698 podStartE2EDuration="19.847575147s" totalImagesPullingTime="10.874783449s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:24.847522216 +0000 UTC m=+19.760999076" watchObservedRunningTime="2026-10-30 22:18:24.847575147 +0000 UTC m=+19.761051994"
Oct 30 22:18:25.606939 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:25.606921    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:25.607037 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:25.606989    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:25.607086 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:25.607071    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:25.607179 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:25.607162    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:26.605690 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:26.605654    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:26.606023 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:26.605776    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:26.824794 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:26.824536    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" event={"ID":"fb7545c5-92b9-4107-93d0-f0617277df0f","Type":"ContainerStarted","Data":"c449c334ba370030a4111769e97c62939c9f2a9bb616e4757a58034cc453fdfc"}
Oct 30 22:18:26.824794 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:26.824764    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:26.839710 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:26.839685    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:26.860340 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:26.860299    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm" podStartSLOduration=7.485346128 podStartE2EDuration="21.860288415s" totalImagesPullingTime="14.374942287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:26.85944222 +0000 UTC m=+21.772919079" watchObservedRunningTime="2026-10-30 22:18:26.860288415 +0000 UTC m=+21.773765263"
Oct 30 22:18:27.606502 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:27.606481    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:27.606930 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:27.606583    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:27.606930 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:27.606688    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:27.606930 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:27.606774    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:27.800552 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:27.800510    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:27.827239 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:27.827212    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:27.844015 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:27.843991    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:28.082093 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:28.082025    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:28.232415 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:28.232390    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:28.411596 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:28.411572    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hgsqv"]
Oct 30 22:18:28.411781 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:28.411678    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:28.411781 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:28.411759    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:28.413894 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:28.413872    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gtwvr"]
Oct 30 22:18:28.414015 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:28.413961    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:28.414071 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:28.414047    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:28.414491 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:28.414472    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4pvll"]
Oct 30 22:18:28.414588 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:28.414562    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:28.414699 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:28.414653    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:29.606680 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:29.606642    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:29.607024 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:29.606738    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:30.606674 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:30.606644    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:30.606822 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:30.606692    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:30.606822 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:30.606767    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:30.607144 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:30.606815    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:30.682329 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:30.682306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:30.782222 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:30.782186    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:30.882180 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:30.882119    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:30.983414 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:30.983389    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:31.605617 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:31.605598    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:31.605768 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:31.605725    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:31.606036 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:31.606002    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:31.706273 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:31.706249    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:32.606537 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:32.606512    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:32.606698 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:32.606515    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:32.606698 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:32.606620    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:32.606814 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:32.606698    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:33.606570 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:33.606544    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:33.607089 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:33.606661    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:33.689805 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:33.689778    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:33.789954 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:33.789926    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:34.181198 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:34.181021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:34.292206 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:34.292152    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:34.413258 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:34.413232    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:34.513890 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:34.513867    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:34.605807 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:34.605788    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:34.605914 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:34.605811    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:34.605973 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:34.605917    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:34.606055 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:34.606035    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:34.616174 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:34.616151    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:34.902238 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:34.902172    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:35.046098 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:35.046070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:35.160569 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:35.160518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:35.400856 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:35.400836    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:35.606676 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:35.606639    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:35.606769 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:35.606733    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:35.697415 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:35.697395    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:36.143237 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:36.143216    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wqsgd_0b6fa19a-6fb7-40b6-9073-0a52cfca6b79/dns-node-resolver/0.log"
Oct 30 22:18:36.538466 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:36.538424    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:36.605598 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:36.605580    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:36.605736 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:36.605600    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:36.605736 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:36.605661    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:36.605822 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:36.605775    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:36.651475 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:36.651451    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:36.761392 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:36.761370    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:36.940922 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:36.940904    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dkmz4_84f3a1bb-9373-443e-b957-1bf6ccb03477/node-ca/0.log"
Oct 30 22:18:36.954512 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:36.954493    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:37.606484 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:37.606465    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:37.606568 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:37.606548    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:38.164454 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:38.164419    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:38.356428 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:38.356405    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:38.569873 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:38.569828    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:38.605991 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:38.605967    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:38.606070 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:38.605999    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:38.606108 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:38.606072    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:38.606154 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:38.606118    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:38.771407 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:38.771384    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:38.977473 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:38.977454    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:39.288077 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:39.288034    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:39.288077 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:39.288066    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:39.288077 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:39.288087    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:39.288451 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:39.288181    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:39.288451 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:39.288181    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:39.288451 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:39.288218    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs podName:e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:11.288207296 +0000 UTC m=+66.201684123 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs") pod "network-metrics-daemon-4pvll" (UID: "e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:39.288451 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:39.288238    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret podName:1ae3621a-b431-4675-8617-37461922d241 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:11.288222991 +0000 UTC m=+66.201699818 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret") pod "global-pull-secret-syncer-hgsqv" (UID: "1ae3621a-b431-4675-8617-37461922d241") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:39.288451 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:39.288244    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:39.288451 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:39.288254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:39.288451 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:39.288267    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:39.288451 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:39.288277    2571 projected.go:196] Error preparing data for projected volume kube-api-access-xgg87 for pod openshift-network-diagnostics/network-check-target-gtwvr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:39.288451 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:39.288328    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87 podName:7cd8a3d9-fbbb-43c0-9920-345d2ab627c3 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:11.288316302 +0000 UTC m=+66.201793142 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xgg87" (UniqueName: "kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87") pod "network-check-target-gtwvr" (UID: "7cd8a3d9-fbbb-43c0-9920-345d2ab627c3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:39.605842 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:39.605825    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:39.605916 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:39.605900    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:39.740631 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:39.740610    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.160526 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:40.160504    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.363751 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:40.363726    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.474620 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:40.474570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.592383 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:40.592362    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.606586 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:40.606567    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:40.606713 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:40.606695    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:40.606775 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:40.606754    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:40.606872 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:40.606855    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:40.754373 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:40.754323    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:40.954709 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:40.954690    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:41.606071 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:41.606050    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:41.606340 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:41.606121    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:42.154798 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:42.154774    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.348484 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:42.348466    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.551527 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:42.551424    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.606009 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:42.605982    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:42.606149 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:42.606086    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:42.606149 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:42.606144    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:42.606461 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:42.606239    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:42.752126 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:42.752065    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:42.947028 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:42.946995    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:43.265530 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:43.265469    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:43.605841 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:43.605821    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:43.606012 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:43.605927    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:44.148426 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:44.148370    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:44.345546 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:44.345518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:44.594733 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:44.594703    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:44.605801 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:44.605782    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:44.605861 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:44.605809    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:44.605899 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:44.605887    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:44.606021 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:44.606000    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:44.789861 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:44.789839    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:44.952156 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:44.952108    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:45.224579 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:45.224541    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:45.606733 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:45.606712    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:45.606829 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:45.606780    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:45.680600 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:45.680579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:45.778821 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:45.778801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:46.148539 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:46.148523    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:46.307325 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:46.307305    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:46.409525 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:46.409484    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:46.528807 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:46.528782    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:46.606614 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:46.606594    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:46.606709 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:46.606695    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:46.606783 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:46.606770    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:46.606857 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:46.606843    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:46.629247 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:46.629231    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:46.751151 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:46.751109    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:46.946219 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:46.946202    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:47.372094 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:47.372074    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:47.606000 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:47.605977    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:47.606112 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:47.606083    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:48.099866 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:48.099830    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:48.606055 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:48.606030    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:48.606475 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:48.606060    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:48.606475 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:48.606153    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:48.606475 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:48.606253    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:49.606474 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:49.606449    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:49.606874 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:49.606544    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:50.605966 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:50.605940    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:50.606104 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:50.606016    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:50.606176 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:50.606106    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:50.606232 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:50.606214    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:51.606477 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:51.606458    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:51.606800 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:51.606531    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:52.605792 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:52.605767    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:52.605952 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:52.605878    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:52.605952 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:52.605878    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:52.606033 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:52.605949    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:52.948961 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:52.948889    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:53.053224 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:53.053189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:53.157947 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:53.157923    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:53.356826 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:53.356792    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:53.464543 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:53.464525    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:53.605831 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:53.605814    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:53.605901 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:53.605887    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gtwvr" podUID="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3"
Oct 30 22:18:54.276005 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.275985    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:54.476836 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.476797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:54.606445 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.606422    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:54.606532 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:54.606504    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hgsqv" podUID="1ae3621a-b431-4675-8617-37461922d241"
Oct 30 22:18:54.606611 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.606596    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:54.606739 ip-10-0-128-97 kubenswrapper[2571]: E1030 22:18:54.606722    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4pvll" podUID="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1"
Oct 30 22:18:54.741103 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.741086    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeReady"
Oct 30 22:18:54.741176 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.741165    2571 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 22:18:54.824568 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.824527    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hrpz9"]
Oct 30 22:18:54.826127 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.826109    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:18:54.828893 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.828876    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-96mpc"]
Oct 30 22:18:54.830366 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.830350    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:54.841013 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.840995    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hrpz9"]
Oct 30 22:18:54.846164 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.846147    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 22:18:54.846301 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.846268    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9jqsq\""
Oct 30 22:18:54.846370 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.846291    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 22:18:54.846423 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.846383    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-jfcbv\""
Oct 30 22:18:54.860868 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.860850    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 22:18:54.860976 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.860961    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 22:18:54.861046 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.861033    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 22:18:54.863438 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.863422    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5rttv"]
Oct 30 22:18:54.865298 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.865275    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:54.866953 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.866922    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-96mpc"]
Oct 30 22:18:54.886199 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.886180    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5rttv"]
Oct 30 22:18:54.901129 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.901109    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/376fe364-ccc4-4435-b5e0-bef1e6715b26-tmp-dir\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:54.901205 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.901144    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/376fe364-ccc4-4435-b5e0-bef1e6715b26-config-volume\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:54.901205 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.901168    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86xfb\" (UniqueName: \"kubernetes.io/projected/376fe364-ccc4-4435-b5e0-bef1e6715b26-kube-api-access-86xfb\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:54.901271 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.901255    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/376fe364-ccc4-4435-b5e0-bef1e6715b26-metrics-tls\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:54.901299 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.901280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcj9x\" (UniqueName: \"kubernetes.io/projected/cd11fe2e-49c0-4478-928a-a04ed1f29273-kube-api-access-wcj9x\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:18:54.901326 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.901306    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd11fe2e-49c0-4478-928a-a04ed1f29273-cert\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:18:54.937751 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.937732    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-srblb\""
Oct 30 22:18:54.937751 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.937748    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 22:18:54.947500 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.947485    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 22:18:54.947559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.947526    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 22:18:54.947559 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:54.947537    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 22:18:55.001827 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.001805    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e972ba1-2295-49d0-998b-00b86a1737af-data-volume\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.001881 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.001833    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86xfb\" (UniqueName: \"kubernetes.io/projected/376fe364-ccc4-4435-b5e0-bef1e6715b26-kube-api-access-86xfb\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:55.001881 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.001857    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e972ba1-2295-49d0-998b-00b86a1737af-crio-socket\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.001881 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.001871    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e972ba1-2295-49d0-998b-00b86a1737af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.001977 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.001950    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/376fe364-ccc4-4435-b5e0-bef1e6715b26-metrics-tls\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:55.002036 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.002022    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcj9x\" (UniqueName: \"kubernetes.io/projected/cd11fe2e-49c0-4478-928a-a04ed1f29273-kube-api-access-wcj9x\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:18:55.002073 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.002045    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wwpn\" (UniqueName: \"kubernetes.io/projected/4e972ba1-2295-49d0-998b-00b86a1737af-kube-api-access-5wwpn\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.002112 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.002071    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd11fe2e-49c0-4478-928a-a04ed1f29273-cert\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:18:55.002146 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.002111    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e972ba1-2295-49d0-998b-00b86a1737af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.002224 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.002214    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/376fe364-ccc4-4435-b5e0-bef1e6715b26-tmp-dir\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:55.002265 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.002240    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/376fe364-ccc4-4435-b5e0-bef1e6715b26-config-volume\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:55.002503 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.002484    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/376fe364-ccc4-4435-b5e0-bef1e6715b26-tmp-dir\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:55.002718 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.002704    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/376fe364-ccc4-4435-b5e0-bef1e6715b26-config-volume\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:55.005804 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.005783    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/376fe364-ccc4-4435-b5e0-bef1e6715b26-metrics-tls\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:55.005872 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.005837    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd11fe2e-49c0-4478-928a-a04ed1f29273-cert\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:18:55.017800 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.017776    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86xfb\" (UniqueName: \"kubernetes.io/projected/376fe364-ccc4-4435-b5e0-bef1e6715b26-kube-api-access-86xfb\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:55.018020 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.018003    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcj9x\" (UniqueName: \"kubernetes.io/projected/cd11fe2e-49c0-4478-928a-a04ed1f29273-kube-api-access-wcj9x\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:18:55.103025 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.103007    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e972ba1-2295-49d0-998b-00b86a1737af-data-volume\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.103095 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.103040    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e972ba1-2295-49d0-998b-00b86a1737af-crio-socket\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.103095 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.103055    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e972ba1-2295-49d0-998b-00b86a1737af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.103095 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.103072    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wwpn\" (UniqueName: \"kubernetes.io/projected/4e972ba1-2295-49d0-998b-00b86a1737af-kube-api-access-5wwpn\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.103182 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.103115    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e972ba1-2295-49d0-998b-00b86a1737af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.103182 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.103170    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e972ba1-2295-49d0-998b-00b86a1737af-crio-socket\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.103394 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.103373    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e972ba1-2295-49d0-998b-00b86a1737af-data-volume\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.103621 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.103599    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e972ba1-2295-49d0-998b-00b86a1737af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.105027 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.105011    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e972ba1-2295-49d0-998b-00b86a1737af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.115332 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.115311    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wwpn\" (UniqueName: \"kubernetes.io/projected/4e972ba1-2295-49d0-998b-00b86a1737af-kube-api-access-5wwpn\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.137252 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.137231    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:18:55.143480 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.143461    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:55.181061 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.181032    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:18:55.418922 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.418819    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hrpz9"]
Oct 30 22:18:55.421050 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.421028    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-96mpc"]
Oct 30 22:18:55.421551 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:55.421522    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd11fe2e_49c0_4478_928a_a04ed1f29273.slice/crio-3e301a910925891def07f423a9e9350fc65d9dc0b259d5d54e710fa2474869b7 WatchSource:0}: Error finding container 3e301a910925891def07f423a9e9350fc65d9dc0b259d5d54e710fa2474869b7: Status 404 returned error can't find the container with id 3e301a910925891def07f423a9e9350fc65d9dc0b259d5d54e710fa2474869b7
Oct 30 22:18:55.423931 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:55.423904    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376fe364_ccc4_4435_b5e0_bef1e6715b26.slice/crio-376987537f0da72e3f9ddeb28370679e49440d37b4ba82e54697db5b8ff7981a WatchSource:0}: Error finding container 376987537f0da72e3f9ddeb28370679e49440d37b4ba82e54697db5b8ff7981a: Status 404 returned error can't find the container with id 376987537f0da72e3f9ddeb28370679e49440d37b4ba82e54697db5b8ff7981a
Oct 30 22:18:55.460464 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.460434    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5rttv"]
Oct 30 22:18:55.463691 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:55.463651    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e972ba1_2295_49d0_998b_00b86a1737af.slice/crio-1267cd9cb859f3320e7b5ef1ff38b43344b6f35a52ce78ea2d86b48dfd541210 WatchSource:0}: Error finding container 1267cd9cb859f3320e7b5ef1ff38b43344b6f35a52ce78ea2d86b48dfd541210: Status 404 returned error can't find the container with id 1267cd9cb859f3320e7b5ef1ff38b43344b6f35a52ce78ea2d86b48dfd541210
Oct 30 22:18:55.608102 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.608073    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:18:55.612598 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.612562    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 22:18:55.612598 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.612588    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 22:18:55.613201 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.613188    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-sdfwl\""
Oct 30 22:18:55.679718 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.679655    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:55.778879 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.778856    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:55.881342 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.881316    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5rttv" event={"ID":"4e972ba1-2295-49d0-998b-00b86a1737af","Type":"ContainerStarted","Data":"2ca47ba5584eb27101a23523f23ec5ac7e9f43620741d34332817201c271322f"}
Oct 30 22:18:55.881455 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.881365    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5rttv" event={"ID":"4e972ba1-2295-49d0-998b-00b86a1737af","Type":"ContainerStarted","Data":"1267cd9cb859f3320e7b5ef1ff38b43344b6f35a52ce78ea2d86b48dfd541210"}
Oct 30 22:18:55.882609 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.882581    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hrpz9" event={"ID":"cd11fe2e-49c0-4478-928a-a04ed1f29273","Type":"ContainerStarted","Data":"3e301a910925891def07f423a9e9350fc65d9dc0b259d5d54e710fa2474869b7"}
Oct 30 22:18:55.883888 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:55.883866    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-96mpc" event={"ID":"376fe364-ccc4-4435-b5e0-bef1e6715b26","Type":"ContainerStarted","Data":"376987537f0da72e3f9ddeb28370679e49440d37b4ba82e54697db5b8ff7981a"}
Oct 30 22:18:56.606601 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:56.606569    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:18:56.607066 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:56.606617    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:18:56.612222 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:56.612199    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 22:18:56.612329 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:56.612204    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms5nl\""
Oct 30 22:18:56.612329 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:56.612242    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 22:18:56.612329 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:56.612249    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-r4dw4\""
Oct 30 22:18:56.893385 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:56.893226    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5rttv" event={"ID":"4e972ba1-2295-49d0-998b-00b86a1737af","Type":"ContainerStarted","Data":"d39b13e6db6854b5778b4d5ef43d2cf2b57a769b1228aaecaa8bdd473968949b"}
Oct 30 22:18:57.578332 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.578221    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:57.601869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.600988    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"]
Oct 30 22:18:57.602973 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.602947    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.607427 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.607320    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 22:18:57.607427 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.607415    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 22:18:57.608431 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.608409    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-l75wj\""
Oct 30 22:18:57.608561 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.608441    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 22:18:57.608561 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.608489    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 22:18:57.608710 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.608693    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 22:18:57.616212 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.616187    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"]
Oct 30 22:18:57.682426 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.682392    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:57.725638 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.725606    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.725785 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.725650    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a410d5a1-3091-405c-a1b2-9d790b77b458-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.725785 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.725695    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwgtv\" (UniqueName: \"kubernetes.io/projected/a410d5a1-3091-405c-a1b2-9d790b77b458-kube-api-access-pwgtv\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.725887 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.725826    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.820372 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.820344    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:57.826538 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.826516    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.826639 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.826545    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a410d5a1-3091-405c-a1b2-9d790b77b458-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.826639 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.826561    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwgtv\" (UniqueName: \"kubernetes.io/projected/a410d5a1-3091-405c-a1b2-9d790b77b458-kube-api-access-pwgtv\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.826639 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.826612    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.827296 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.827278    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a410d5a1-3091-405c-a1b2-9d790b77b458-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.830161 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.830120    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.830220 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.830162    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.840613 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.840587    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwgtv\" (UniqueName: \"kubernetes.io/projected/a410d5a1-3091-405c-a1b2-9d790b77b458-kube-api-access-pwgtv\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.897183 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.897155    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-96mpc" event={"ID":"376fe364-ccc4-4435-b5e0-bef1e6715b26","Type":"ContainerStarted","Data":"3f4afdcb572a49a54be0946654b14f0737e1a762fe535cbb460cf40da1acec73"}
Oct 30 22:18:57.897287 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.897218    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-96mpc" event={"ID":"376fe364-ccc4-4435-b5e0-bef1e6715b26","Type":"ContainerStarted","Data":"827aae8205bcbbc92a89aef07fe476ea7d6932453bdb8677a9afc6c220db1596"}
Oct 30 22:18:57.897414 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.897397    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-96mpc"
Oct 30 22:18:57.899712 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.899689    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5rttv" event={"ID":"4e972ba1-2295-49d0-998b-00b86a1737af","Type":"ContainerStarted","Data":"0b79f16313a43f0c9e3db50fd26a2eb9cfd6b78ace78db8cdfa224e5afe01ebf"}
Oct 30 22:18:57.917796 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.917775    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:18:57.918790 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.918740    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-96mpc" podStartSLOduration=2.035326007 podStartE2EDuration="3.918725733s" totalImagesPullingTime="1.883399726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:57.91797234 +0000 UTC m=+52.831449193" watchObservedRunningTime="2026-10-30 22:18:57.918725733 +0000 UTC m=+52.832202581"
Oct 30 22:18:57.939813 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:57.939615    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5rttv" podStartSLOduration=1.7243829179999999 podStartE2EDuration="3.939602259s" totalImagesPullingTime="2.215219341s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:57.938617268 +0000 UTC m=+52.852094116" watchObservedRunningTime="2026-10-30 22:18:57.939602259 +0000 UTC m=+52.853079106"
Oct 30 22:18:58.193073 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:58.193044    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"]
Oct 30 22:18:58.195821 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:18:58.195791    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda410d5a1_3091_405c_a1b2_9d790b77b458.slice/crio-533726c955f944a49a6efe13c0d836e8f1bf9689485cc747ea2b65cb09ef8962 WatchSource:0}: Error finding container 533726c955f944a49a6efe13c0d836e8f1bf9689485cc747ea2b65cb09ef8962: Status 404 returned error can't find the container with id 533726c955f944a49a6efe13c0d836e8f1bf9689485cc747ea2b65cb09ef8962
Oct 30 22:18:58.903832 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:58.903758    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62" event={"ID":"a410d5a1-3091-405c-a1b2-9d790b77b458","Type":"ContainerStarted","Data":"533726c955f944a49a6efe13c0d836e8f1bf9689485cc747ea2b65cb09ef8962"}
Oct 30 22:18:58.905443 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:58.905414    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hrpz9" event={"ID":"cd11fe2e-49c0-4478-928a-a04ed1f29273","Type":"ContainerStarted","Data":"097d1b14c9614219ca2fb5f5ac3439802b4a6fcf986a48f47dd4bdccbaa1aeb3"}
Oct 30 22:18:58.924401 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:58.924346    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hrpz9" podStartSLOduration=2.271661608 podStartE2EDuration="4.924330956s" totalImagesPullingTime="2.652669348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:58.924246412 +0000 UTC m=+53.837723271" watchObservedRunningTime="2026-10-30 22:18:58.924330956 +0000 UTC m=+53.837807803"
Oct 30 22:18:59.671357 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:59.671330    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:18:59.841384 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:59.841363    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:18:59.910170 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:59.910139    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62" event={"ID":"a410d5a1-3091-405c-a1b2-9d790b77b458","Type":"ContainerStarted","Data":"638899c07d1fbe5e9164ddc14cc106d745b37f6862c7c46248bb4f95365d2d27"}
Oct 30 22:18:59.910486 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:59.910185    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62" event={"ID":"a410d5a1-3091-405c-a1b2-9d790b77b458","Type":"ContainerStarted","Data":"7bf7fc4989f295ae97c6eef9d88e16034da49742d50a5f98d0d9cb04ab2f2ccc"}
Oct 30 22:18:59.931123 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:18:59.931049    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62" podStartSLOduration=1.82491228 podStartE2EDuration="2.931033331s" totalImagesPullingTime="1.106121051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:59.93043538 +0000 UTC m=+54.843912227" watchObservedRunningTime="2026-10-30 22:18:59.931033331 +0000 UTC m=+54.844510180"
Oct 30 22:19:00.682148 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:00.682122    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:00.778039 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:00.778012    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:01.772185 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:01.772154    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:01.873977 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:01.873953    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.027742 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.027688    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jxc4l"]
Oct 30 22:19:02.030929 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.030907    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.057253 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.057077    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 22:19:02.057865 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.057845    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.058074 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.058057    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 22:19:02.058119 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.058077    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bh4rp\""
Oct 30 22:19:02.129827 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.129804    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 22:19:02.157076 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.157052    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-root\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.157159 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.157087    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.157159 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.157119    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-textfile\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.157245 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.157212    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdx5q\" (UniqueName: \"kubernetes.io/projected/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-kube-api-access-sdx5q\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.157245 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.157236    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-sys\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.157318 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.157283    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.157404 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.157388    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-tls\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.157460 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.157431    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-metrics-client-ca\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.157532 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.157516    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-wtmp\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.168122 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.168100    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.258609 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258554    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-tls\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258609 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258610    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-metrics-client-ca\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-wtmp\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258697    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-root\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258715    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-textfile\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258778    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdx5q\" (UniqueName: \"kubernetes.io/projected/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-kube-api-access-sdx5q\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258806    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-sys\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258804    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-root\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258853    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.258869 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258861    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-wtmp\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.259217 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.258879    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-sys\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.259272 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.259215    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-textfile\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.259365 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.259347    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-metrics-client-ca\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.259502 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.259376    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.261203 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.261182    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-tls\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.261300 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.261218    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.272874 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.272849    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdx5q\" (UniqueName: \"kubernetes.io/projected/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-kube-api-access-sdx5q\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.366366 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.366328    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:02.378765 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:19:02.378734    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ecaf39f_ddd9_488f_9c8a_8fc7664eb2f2.slice/crio-184da35d5d7690b39f4735a6c51c7b9a264784a04e691d74fe0a0733a5b6a684 WatchSource:0}: Error finding container 184da35d5d7690b39f4735a6c51c7b9a264784a04e691d74fe0a0733a5b6a684: Status 404 returned error can't find the container with id 184da35d5d7690b39f4735a6c51c7b9a264784a04e691d74fe0a0733a5b6a684
Oct 30 22:19:02.416726 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.416697    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.517326 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.517291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.633695 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.633616    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:02.919756 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:02.919681    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jxc4l" event={"ID":"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2","Type":"ContainerStarted","Data":"184da35d5d7690b39f4735a6c51c7b9a264784a04e691d74fe0a0733a5b6a684"}
Oct 30 22:19:03.050405 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:03.050367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:03.164938 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:03.164907    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:03.364440 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:03.364403    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:04.023376 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.023351    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:04.125090 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.125068    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6776c6b778-2gtz5"]
Oct 30 22:19:04.128166 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.128151    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.133327 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.133306    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 22:19:04.143944 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.143878    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6776c6b778-2gtz5"]
Oct 30 22:19:04.148327 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.148301    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:04.233223 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.233198    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 22:19:04.233312 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.233258    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 22:19:04.233312 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.233283    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hx8r9\""
Oct 30 22:19:04.233396 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.233377    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 22:19:04.233428 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.233399    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fal81smpff8pm\""
Oct 30 22:19:04.234047 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.234035    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 22:19:04.274319 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.274293    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.274396 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.274333    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-grpc-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.274396 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.274353    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-tmp\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.274459 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.274410    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.274492 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.274463    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.274541 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.274523    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.274599 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.274587    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.274632 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.274606    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89prq\" (UniqueName: \"kubernetes.io/projected/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-kube-api-access-89prq\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.274707 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.274691    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-metrics-client-ca\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.375828 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.375800    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.375884 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.375833    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.375932 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.375872    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.375932 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.375917    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89prq\" (UniqueName: \"kubernetes.io/projected/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-kube-api-access-89prq\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.376016 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.375949    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-metrics-client-ca\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.376016 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.375975    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.376132 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.376109    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-grpc-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.376205 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.376154    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-tmp\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.376259 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.376200    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.376772 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.376697    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-tmp\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.377114 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.376901    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-metrics-client-ca\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.378657 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.378631    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.378794 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.378772    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.378865 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.378792    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-grpc-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.378865 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.378830    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.378967 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.378952    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.379019 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.379004    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.390331 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.390309    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89prq\" (UniqueName: \"kubernetes.io/projected/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-kube-api-access-89prq\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.448993 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.448953    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:04.730832 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.730766    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6776c6b778-2gtz5"]
Oct 30 22:19:04.734501 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:19:04.734476    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc754a9ec_30eb_4e30_a6ca_f2f2f23fe87f.slice/crio-dc1164eca5be445f995c88bd2c858c3155e577684f706032dbf418a2c09e103d WatchSource:0}: Error finding container dc1164eca5be445f995c88bd2c858c3155e577684f706032dbf418a2c09e103d: Status 404 returned error can't find the container with id dc1164eca5be445f995c88bd2c858c3155e577684f706032dbf418a2c09e103d
Oct 30 22:19:04.925838 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.925810    2571 generic.go:356] "Generic (PLEG): container finished" podID="2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2" containerID="d776ac4f78b5b757604701f0a9a8c850d2ed9d85bf25efa56ebfb61b706b6a45" exitCode=0
Oct 30 22:19:04.925929 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.925898    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jxc4l" event={"ID":"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2","Type":"ContainerDied","Data":"d776ac4f78b5b757604701f0a9a8c850d2ed9d85bf25efa56ebfb61b706b6a45"}
Oct 30 22:19:04.927065 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:04.927048    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5" event={"ID":"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f","Type":"ContainerStarted","Data":"dc1164eca5be445f995c88bd2c858c3155e577684f706032dbf418a2c09e103d"}
Oct 30 22:19:05.095050 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:05.095024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:05.200127 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:05.200102    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:05.320297 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:05.320220    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:05.424480 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:05.424450    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:05.758080 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:05.758000    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:05.857189 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:05.857154    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:05.932656 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:05.932626    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jxc4l" event={"ID":"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2","Type":"ContainerStarted","Data":"40eb8a426792d0200f6f86978a7041473ed057b581caf02588f57066a916f9e2"}
Oct 30 22:19:05.932827 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:05.932701    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jxc4l" event={"ID":"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2","Type":"ContainerStarted","Data":"254e1fa397b89bc32b6898576e879609f403f43f8f2105a789d8a404f7ff97de"}
Oct 30 22:19:05.960548 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:05.960501    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jxc4l" podStartSLOduration=3.3810475159999998 podStartE2EDuration="4.960488118s" totalImagesPullingTime="1.508682809s" totalInitContainerRuntime="70.757793ms" isStatefulPod=true podCreationTimestamp="2026-10-30 22:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:05.958505242 +0000 UTC m=+60.871982094" watchObservedRunningTime="2026-10-30 22:19:05.960488118 +0000 UTC m=+60.873964965"
Oct 30 22:19:06.434887 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.434853    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5f5784d99-gnbws"]
Oct 30 22:19:06.437982 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.437962    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.446102 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.446081    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 22:19:06.446236 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.446080    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 22:19:06.453151 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.452279    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f5784d99-gnbws"]
Oct 30 22:19:06.463322 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.463296    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 22:19:06.463479 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.463461    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 22:19:06.463479 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.463474    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2j6f8qkq0ohq5\""
Oct 30 22:19:06.463616 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.463500    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dncrj\""
Oct 30 22:19:06.595635 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.595594    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-tls\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.595812 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.595648    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-client-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.595812 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.595783    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-metrics-server-audit-profiles\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.595937 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.595852    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-client-certs\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.595937 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.595884    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.595937 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.595923    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8db49c21-a7b4-4fc6-8342-95243844e22f-audit-log\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.596084 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.595953    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtppr\" (UniqueName: \"kubernetes.io/projected/8db49c21-a7b4-4fc6-8342-95243844e22f-kube-api-access-xtppr\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.697168 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.697098    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-client-certs\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.697168 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.697134    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.697168 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.697158    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8db49c21-a7b4-4fc6-8342-95243844e22f-audit-log\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.697385 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.697177    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtppr\" (UniqueName: \"kubernetes.io/projected/8db49c21-a7b4-4fc6-8342-95243844e22f-kube-api-access-xtppr\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.697385 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.697207    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-tls\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.697385 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.697232    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-client-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.697385 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.697262    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-metrics-server-audit-profiles\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.697601 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.697578    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8db49c21-a7b4-4fc6-8342-95243844e22f-audit-log\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.698166 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.698137    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-metrics-server-audit-profiles\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.698276 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.698164    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.699840 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.699814    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-client-certs\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.699926 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.699816    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-tls\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.700003 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.699973    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-client-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.710851 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.710829    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtppr\" (UniqueName: \"kubernetes.io/projected/8db49c21-a7b4-4fc6-8342-95243844e22f-kube-api-access-xtppr\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.750598 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.750523    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:06.808056 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.807851    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:06.811992 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.810706    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-249ck"]
Oct 30 22:19:06.816700 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.816349    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:06.827476 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.827451    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 22:19:06.828233 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.828215    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lrk6p\""
Oct 30 22:19:06.829243 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.829220    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-249ck"]
Oct 30 22:19:06.915848 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.915801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:06.936982 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.936952    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5" event={"ID":"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f","Type":"ContainerStarted","Data":"99f89d2d3291417210fd5293483ea8e6c2d6a939c74cf96886c439fad6d48201"}
Oct 30 22:19:06.937079 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.937036    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5" event={"ID":"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f","Type":"ContainerStarted","Data":"e9ce26bcc9655737e1541c66aed698e1678f5e52b5c2a4893796cf668b3a0224"}
Oct 30 22:19:06.937079 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.937062    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5" event={"ID":"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f","Type":"ContainerStarted","Data":"d49285ba0fe253eb23cb8e0e91c9028aa53bbafef1c56a0612cbe10eebbc35cd"}
Oct 30 22:19:06.999609 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.999544    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9115c245-4790-4124-a435-24b667fc1744-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:06.999743 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:06.999610    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq6r9\" (UniqueName: \"kubernetes.io/projected/9115c245-4790-4124-a435-24b667fc1744-kube-api-access-zq6r9\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:07.021194 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.021171    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:07.033311 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.033286    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5f5784d99-gnbws"]
Oct 30 22:19:07.035811 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:19:07.035791    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db49c21_a7b4_4fc6_8342_95243844e22f.slice/crio-1594ec48944c128dd92eb89f8bf470a5da5a0419b05610f83939c3bd6dcdad58 WatchSource:0}: Error finding container 1594ec48944c128dd92eb89f8bf470a5da5a0419b05610f83939c3bd6dcdad58: Status 404 returned error can't find the container with id 1594ec48944c128dd92eb89f8bf470a5da5a0419b05610f83939c3bd6dcdad58
Oct 30 22:19:07.100451 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.100414    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq6r9\" (UniqueName: \"kubernetes.io/projected/9115c245-4790-4124-a435-24b667fc1744-kube-api-access-zq6r9\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:07.100614 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.100519    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9115c245-4790-4124-a435-24b667fc1744-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:07.102884 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.102860    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9115c245-4790-4124-a435-24b667fc1744-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:07.114212 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.114192    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq6r9\" (UniqueName: \"kubernetes.io/projected/9115c245-4790-4124-a435-24b667fc1744-kube-api-access-zq6r9\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:07.133414 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.133394    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:07.247490 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.247462    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-56c84f9555-d5rqg"]
Oct 30 22:19:07.251654 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.251614    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.256506 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.256482    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 22:19:07.256622 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.256589    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 22:19:07.256759 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.256739    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 22:19:07.257435 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.257419    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w24gg\""
Oct 30 22:19:07.257603 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.257591    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 22:19:07.257897 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.257881    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 22:19:07.262472 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.262453    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 22:19:07.267964 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.267937    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56c84f9555-d5rqg"]
Oct 30 22:19:07.402701 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.402651    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-federate-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.402944 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.402721    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.402944 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.402814    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.402944 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.402923    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-metrics-client-ca\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.403519 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.402965    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.403519 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.403062    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.403519 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.403131    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmk4b\" (UniqueName: \"kubernetes.io/projected/9701f603-d98d-49b5-adc7-3550bdab41d2-kube-api-access-rmk4b\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.403519 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.403173    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-serving-certs-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.404192 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.404168    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-249ck"]
Oct 30 22:19:07.407844 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:19:07.407822    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9115c245_4790_4124_a435_24b667fc1744.slice/crio-0eb3434f123f1a6456f8503a2c7b791fc092551e23f2b2419be529aa78b77868 WatchSource:0}: Error finding container 0eb3434f123f1a6456f8503a2c7b791fc092551e23f2b2419be529aa78b77868: Status 404 returned error can't find the container with id 0eb3434f123f1a6456f8503a2c7b791fc092551e23f2b2419be529aa78b77868
Oct 30 22:19:07.503756 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.503700    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.503756 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.503733    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.504194 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.503780    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmk4b\" (UniqueName: \"kubernetes.io/projected/9701f603-d98d-49b5-adc7-3550bdab41d2-kube-api-access-rmk4b\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.504194 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.503814    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-serving-certs-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.504194 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.503907    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-federate-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.504194 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.503941    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.504194 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.503977    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.504194 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.504004    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-metrics-client-ca\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.504733 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.504691    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-metrics-client-ca\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.505159 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.505139    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-serving-certs-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.505720 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.505698    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.506480 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.506446    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.506860 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.506839    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-federate-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.506938 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.506843    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.507320 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.507304    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.516676 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.516646    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmk4b\" (UniqueName: \"kubernetes.io/projected/9701f603-d98d-49b5-adc7-3550bdab41d2-kube-api-access-rmk4b\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.567269 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.567242    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:07.851045 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.850841    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-56c84f9555-d5rqg"]
Oct 30 22:19:07.855195 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:19:07.855173    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9701f603_d98d_49b5_adc7_3550bdab41d2.slice/crio-aa75e6408b852ec372edc27d5e83df8542d3250cdfc227111b442e043141b976 WatchSource:0}: Error finding container aa75e6408b852ec372edc27d5e83df8542d3250cdfc227111b442e043141b976: Status 404 returned error can't find the container with id aa75e6408b852ec372edc27d5e83df8542d3250cdfc227111b442e043141b976
Oct 30 22:19:07.908556 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.907928    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:07.955547 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.955494    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck" event={"ID":"9115c245-4790-4124-a435-24b667fc1744","Type":"ContainerStarted","Data":"0eb3434f123f1a6456f8503a2c7b791fc092551e23f2b2419be529aa78b77868"}
Oct 30 22:19:07.956827 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.956634    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f5784d99-gnbws" event={"ID":"8db49c21-a7b4-4fc6-8342-95243844e22f","Type":"ContainerStarted","Data":"1594ec48944c128dd92eb89f8bf470a5da5a0419b05610f83939c3bd6dcdad58"}
Oct 30 22:19:07.957737 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.957695    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg" event={"ID":"9701f603-d98d-49b5-adc7-3550bdab41d2","Type":"ContainerStarted","Data":"aa75e6408b852ec372edc27d5e83df8542d3250cdfc227111b442e043141b976"}
Oct 30 22:19:07.961014 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:07.960916    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5" event={"ID":"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f","Type":"ContainerStarted","Data":"483426febeb3658dce7117395bafe60f9def278f6d8ed3c38d6c7354ba9bd747"}
Oct 30 22:19:08.478083 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:08.478024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:08.913326 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:08.913289    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:08.966543 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:08.966510    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5f5784d99-gnbws" event={"ID":"8db49c21-a7b4-4fc6-8342-95243844e22f","Type":"ContainerStarted","Data":"735ba416c62ea158ff7581cb289d6108cc040d47d773341badc0b215e9ef8ad2"}
Oct 30 22:19:08.970781 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:08.970750    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5" event={"ID":"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f","Type":"ContainerStarted","Data":"15e0c72c6d14d397fe5a1d8c487a697181be41fdb2742532629e37608eca8de9"}
Oct 30 22:19:08.970923 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:08.970826    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5" event={"ID":"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f","Type":"ContainerStarted","Data":"cdc0718321e329fad881b257a320628a20930dedf5aa6433ed109ae83fcff36f"}
Oct 30 22:19:08.971115 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:08.971097    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:08.991820 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:08.991779    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5f5784d99-gnbws" podStartSLOduration=1.307176403 podStartE2EDuration="2.991766996s" totalImagesPullingTime="1.684590593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:08.990051354 +0000 UTC m=+63.903528197" watchObservedRunningTime="2026-10-30 22:19:08.991766996 +0000 UTC m=+63.905243846"
Oct 30 22:19:09.020813 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:09.020743    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5" podStartSLOduration=2.111584967 podStartE2EDuration="5.020730983s" totalImagesPullingTime="2.909146016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:09.018631933 +0000 UTC m=+63.932108780" watchObservedRunningTime="2026-10-30 22:19:09.020730983 +0000 UTC m=+63.934207829"
Oct 30 22:19:09.976416 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:09.976319    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck" event={"ID":"9115c245-4790-4124-a435-24b667fc1744","Type":"ContainerStarted","Data":"1af11df6a6343233e7dedb1a006e4552d37f722fdf7cfcc8ec2706bd10be3830"}
Oct 30 22:19:09.976854 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:09.976496    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:09.978172 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:09.978140    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg" event={"ID":"9701f603-d98d-49b5-adc7-3550bdab41d2","Type":"ContainerStarted","Data":"9473fb5efd94d89790aeb672006a67e2d8f32f8f5d5f4ac5f22d69148078df4a"}
Oct 30 22:19:09.982225 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:09.982204    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:09.996174 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:09.996123    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck" podStartSLOduration=2.347282528 podStartE2EDuration="3.996108669s" totalImagesPullingTime="1.648826141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:09.995123167 +0000 UTC m=+64.908600016" watchObservedRunningTime="2026-10-30 22:19:09.996108669 +0000 UTC m=+64.909585516"
Oct 30 22:19:10.524798 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:10.524766    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:10.635831 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:10.635806    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:10.741868 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:10.741835    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:10.983435 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:10.983409    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg" event={"ID":"9701f603-d98d-49b5-adc7-3550bdab41d2","Type":"ContainerStarted","Data":"a15ef11d53755b44a0b60b25b0648667c10be0e38c69a0c67b0e700f39b5497f"}
Oct 30 22:19:10.983848 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:10.983505    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg" event={"ID":"9701f603-d98d-49b5-adc7-3550bdab41d2","Type":"ContainerStarted","Data":"0da5c96b763823160faa8178aecda5c66c50f20e5f5ec02ce601e37064e52657"}
Oct 30 22:19:11.012218 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.012175    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg" podStartSLOduration=1.323773952 podStartE2EDuration="4.012160319s" totalImagesPullingTime="2.688386367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:11.010086192 +0000 UTC m=+65.923563033" watchObservedRunningTime="2026-10-30 22:19:11.012160319 +0000 UTC m=+65.925637168"
Oct 30 22:19:11.345374 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.344644    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:19:11.345374 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.344745    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:19:11.345374 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.344802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:11.347619 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.347600    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 22:19:11.347728 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.347715    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 22:19:11.347769 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.347753    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 22:19:11.358442 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.358425    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 22:19:11.358524 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.358504    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:11.358617 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.358603    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:19:11.368806 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.368783    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:19:11.520633 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.520606    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-sdfwl\""
Oct 30 22:19:11.528738 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.528712    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:19:11.625283 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.625223    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms5nl\""
Oct 30 22:19:11.630278 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.630253    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-r4dw4\""
Oct 30 22:19:11.634427 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.634210    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:19:11.637791 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.637768    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:11.802720 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.802696    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gtwvr"]
Oct 30 22:19:11.805261 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:19:11.805234    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cd8a3d9_fbbb_43c0_9920_345d2ab627c3.slice/crio-c9241f3113201235de7daaefcfde220c853f7de35113b0df014bdf59d39e88af WatchSource:0}: Error finding container c9241f3113201235de7daaefcfde220c853f7de35113b0df014bdf59d39e88af: Status 404 returned error can't find the container with id c9241f3113201235de7daaefcfde220c853f7de35113b0df014bdf59d39e88af
Oct 30 22:19:11.918990 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.918941    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hgsqv"]
Oct 30 22:19:11.921285 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:19:11.921258    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae3621a_b431_4675_8617_37461922d241.slice/crio-67fa66cc090e754099433c576545e02bd6b4f736fa6c2df10b46f01eb9e9ed35 WatchSource:0}: Error finding container 67fa66cc090e754099433c576545e02bd6b4f736fa6c2df10b46f01eb9e9ed35: Status 404 returned error can't find the container with id 67fa66cc090e754099433c576545e02bd6b4f736fa6c2df10b46f01eb9e9ed35
Oct 30 22:19:11.922423 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.922405    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4pvll"]
Oct 30 22:19:11.924526 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:19:11.924502    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode27e04a7_2065_4d2e_84fc_4dd5eeabb6c1.slice/crio-b1abc849912eaceb78025dbe82def1660754cf121c3aa234f6343ee0e9f10148 WatchSource:0}: Error finding container b1abc849912eaceb78025dbe82def1660754cf121c3aa234f6343ee0e9f10148: Status 404 returned error can't find the container with id b1abc849912eaceb78025dbe82def1660754cf121c3aa234f6343ee0e9f10148
Oct 30 22:19:11.987759 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.987733    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hgsqv" event={"ID":"1ae3621a-b431-4675-8617-37461922d241","Type":"ContainerStarted","Data":"67fa66cc090e754099433c576545e02bd6b4f736fa6c2df10b46f01eb9e9ed35"}
Oct 30 22:19:11.988812 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.988787    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gtwvr" event={"ID":"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3","Type":"ContainerStarted","Data":"c9241f3113201235de7daaefcfde220c853f7de35113b0df014bdf59d39e88af"}
Oct 30 22:19:11.989764 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:11.989745    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4pvll" event={"ID":"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1","Type":"ContainerStarted","Data":"b1abc849912eaceb78025dbe82def1660754cf121c3aa234f6343ee0e9f10148"}
Oct 30 22:19:12.259971 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.259901    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7ddcd679db-gbd7j"]
Oct 30 22:19:12.262992 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.262975    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.267473 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.267453    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 22:19:12.267566 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.267499    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 22:19:12.267706 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.267648    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 22:19:12.267789 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.267705    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 22:19:12.267789 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.267710    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 22:19:12.267789 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.267726    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 22:19:12.267789 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.267736    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gb6fw\""
Oct 30 22:19:12.267789 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.267756    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 22:19:12.273154 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.273089    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 22:19:12.274372 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.274353    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddcd679db-gbd7j"]
Oct 30 22:19:12.355478 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.355453    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.355620 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.355495    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48zdx\" (UniqueName: \"kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.355620 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.355531    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.355773 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.355645    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.355773 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.355706    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.355773 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.355761    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.355893 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.355793    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.355893 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.355868    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.456685 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.456637    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.456799 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.456699    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.456799 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.456732    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.456799 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.456760    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.456954 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.456793    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.456954 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.456846    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.456954 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.456876    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48zdx\" (UniqueName: \"kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.456954 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.456911    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.459996 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.457501    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.459996 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.458059    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.459996 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.458568    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.459996 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.458838    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.459996 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.459958    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.461512 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.461487    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.461742 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.461711    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.473578 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.473551    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48zdx\" (UniqueName: \"kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.579207 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.579143    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:12.879211 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.878817    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7ddcd679db-gbd7j"]
Oct 30 22:19:12.881405 ip-10-0-128-97 kubenswrapper[2571]: W1030 22:19:12.881372    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bae499a_6abf_4f3a_8e0b_9bb821884f74.slice/crio-ff634edbd2c1ec04cba1038ff5f6834e80222d86bcdecc21de57862cbb681415 WatchSource:0}: Error finding container ff634edbd2c1ec04cba1038ff5f6834e80222d86bcdecc21de57862cbb681415: Status 404 returned error can't find the container with id ff634edbd2c1ec04cba1038ff5f6834e80222d86bcdecc21de57862cbb681415
Oct 30 22:19:12.996916 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:12.996847    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddcd679db-gbd7j" event={"ID":"3bae499a-6abf-4f3a-8e0b-9bb821884f74","Type":"ContainerStarted","Data":"ff634edbd2c1ec04cba1038ff5f6834e80222d86bcdecc21de57862cbb681415"}
Oct 30 22:19:13.496990 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:13.496647    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:13.597464 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:13.597424    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:13.863962 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:13.863925    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:13.964388 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:13.964352    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:14.003100 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:14.003067    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4pvll" event={"ID":"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1","Type":"ContainerStarted","Data":"08ed2f587451d0e3c4c76a3a7b45b9badf6d58e97e88c7c7889bc254a6b8a13b"}
Oct 30 22:19:14.003531 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:14.003131    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4pvll" event={"ID":"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1","Type":"ContainerStarted","Data":"dc606982b683cda2b53b2827a83f41295ba48d51d644fd7040b7844bad0a07df"}
Oct 30 22:19:14.024224 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:14.023692    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4pvll" podStartSLOduration=67.823051958 podStartE2EDuration="1m9.023660383s" totalImagesPullingTime="1.200608425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:14.022732415 +0000 UTC m=+68.936209262" watchObservedRunningTime="2026-10-30 22:19:14.023660383 +0000 UTC m=+68.937137232"
Oct 30 22:19:14.986967 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:14.986931    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:15.208227 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:15.208186    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:15.342048 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:15.342011    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:15.694417 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:15.694388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:15.846927 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:15.846808    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:16.013535 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:16.013449    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gtwvr" event={"ID":"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3","Type":"ContainerStarted","Data":"5ad79860849bc8cd966ba1ec60ad722dce8dbb1e8dbc3a07d85d6e2abc7f4915"}
Oct 30 22:19:16.013726 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:16.013589    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:19:16.665182 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:16.665098    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:16.778311 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:16.778269    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:17.018742 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:17.018641    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddcd679db-gbd7j" event={"ID":"3bae499a-6abf-4f3a-8e0b-9bb821884f74","Type":"ContainerStarted","Data":"2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a"}
Oct 30 22:19:17.051488 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:17.051441    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7ddcd679db-gbd7j" podStartSLOduration=1.569473226 podStartE2EDuration="5.051427604s" totalImagesPullingTime="3.481954378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:17.050087396 +0000 UTC m=+71.963564248" watchObservedRunningTime="2026-10-30 22:19:17.051427604 +0000 UTC m=+71.964904456"
Oct 30 22:19:17.051779 ip-10-0-128-97 kubenswrapper[2571]: I1030 22:19:17.051749    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gtwvr" podStartSLOduration=68.165329919 podStartE2EDuration="1m12.05174007s" totalImagesPullingTime="3.886410151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:16.036690571 +0000 UTC m=+70.950167419" watchObservedRunningTime="2026-10-30 22:19:17.05174007 +0000 UTC m=+71.965216918"
Oct 30 22:19:17.659944 ip-10-0-128-97 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 22:19:17.670037 ip-10-0-128-97 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 22:19:17.670196 ip-10-0-128-97 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 22:19:17.670242 ip-10-0-128-97 systemd[1]: kubelet.service: Consumed 3.826s CPU time.
Oct 30 22:19:17.688122 ip-10-0-128-97 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 22:19:17.762052 ip-10-0-128-97 kubenswrapper[5503]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:19:17.762052 ip-10-0-128-97 kubenswrapper[5503]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 22:19:17.762052 ip-10-0-128-97 kubenswrapper[5503]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:19:17.762052 ip-10-0-128-97 kubenswrapper[5503]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:19:17.764014 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.763934    5503 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:17.764014 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764013    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764017    5503 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764021    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764024    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764027    5503 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764030    5503 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764032    5503 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764035    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764037    5503 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764040    5503 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764043    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764045    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764049    5503 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764052    5503 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764054    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764057    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764059    5503 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764062    5503 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764064    5503 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764067    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:17.764084 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764070    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764072    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764074    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764077    5503 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764079    5503 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764082    5503 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764085    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764087    5503 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764090    5503 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764093    5503 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764096    5503 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764098    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764101    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764103    5503 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764106    5503 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764108    5503 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764111    5503 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764113    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764116    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764118    5503 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:17.764541 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764121    5503 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764123    5503 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764126    5503 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764128    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764131    5503 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764137    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764140    5503 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764142    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764144    5503 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764147    5503 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764149    5503 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764152    5503 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764154    5503 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764157    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764159    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764162    5503 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764164    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764169    5503 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764172    5503 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:17.765035 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764174    5503 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764177    5503 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764179    5503 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764182    5503 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764184    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764186    5503 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764189    5503 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764191    5503 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764195    5503 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764200    5503 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764203    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764207    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764210    5503 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764213    5503 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764215    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764218    5503 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764220    5503 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764223    5503 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764225    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:17.765515 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764233    5503 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764236    5503 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764238    5503 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764241    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764243    5503 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764246    5503 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764250    5503 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764253    5503 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764256    5503 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764258    5503 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764261    5503 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764265    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764267    5503 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764270    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764273    5503 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764276    5503 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764278    5503 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764280    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764283    5503 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:17.765972 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764285    5503 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764288    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764290    5503 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764292    5503 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764295    5503 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764298    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764300    5503 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.764303    5503 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765051    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765058    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765061    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765064    5503 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765067    5503 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765070    5503 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765073    5503 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765076    5503 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765079    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765081    5503 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765083    5503 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765086    5503 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:17.766403 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765089    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765091    5503 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765094    5503 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765096    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765100    5503 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765103    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765105    5503 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765108    5503 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765110    5503 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765113    5503 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765115    5503 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765118    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765120    5503 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765123    5503 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765125    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765130    5503 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765133    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765136    5503 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765139    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:17.767022 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765141    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765144    5503 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765146    5503 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765149    5503 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765151    5503 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765154    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765156    5503 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765158    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765161    5503 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765164    5503 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765166    5503 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765169    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765171    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765174    5503 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765176    5503 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765179    5503 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765181    5503 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765184    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765187    5503 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765190    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:17.767461 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765192    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765195    5503 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765197    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765199    5503 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765202    5503 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765204    5503 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765207    5503 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765209    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765212    5503 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765215    5503 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765218    5503 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765220    5503 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765222    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765225    5503 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765227    5503 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765230    5503 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765232    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765234    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765236    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:17.767976 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765239    5503 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765242    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765244    5503 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765247    5503 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765249    5503 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765252    5503 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765254    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765256    5503 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765259    5503 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765261    5503 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765265    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765267    5503 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765270    5503 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765272    5503 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765275    5503 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765277    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765280    5503 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765282    5503 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765285    5503 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765287    5503 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:17.768414 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765290    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765292    5503 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765295    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765299    5503 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765303    5503 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765307    5503 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765311    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765315    5503 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765319    5503 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765323    5503 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765327    5503 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765331    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765338    5503 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765342    5503 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765345    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765348    5503 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765825    5503 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765830    5503 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765833    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:17.768891 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765836    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765839    5503 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765841    5503 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765845    5503 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765848    5503 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765850    5503 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765853    5503 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765856    5503 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765858    5503 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765861    5503 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765863    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765866    5503 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765869    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765872    5503 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765875    5503 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765877    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765881    5503 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765884    5503 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765887    5503 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765889    5503 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:17.769325 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765892    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765895    5503 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765897    5503 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765900    5503 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765902    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765905    5503 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765907    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765910    5503 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765914    5503 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765919    5503 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765922    5503 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765925    5503 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765928    5503 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765930    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765934    5503 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765938    5503 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765941    5503 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765944    5503 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765946    5503 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:17.770059 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765949    5503 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765951    5503 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765953    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765956    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765958    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765961    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765963    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765966    5503 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765968    5503 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765971    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765973    5503 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765976    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765978    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765981    5503 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765983    5503 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765986    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765988    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765991    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765993    5503 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:17.770870 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765995    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.765998    5503 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766000    5503 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766003    5503 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766005    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766008    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766010    5503 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766013    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766015    5503 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766019    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766021    5503 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766024    5503 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766026    5503 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766028    5503 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766031    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766033    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766036    5503 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766038    5503 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766041    5503 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:17.771329 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766043    5503 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766046    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766048    5503 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766050    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766053    5503 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766055    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766057    5503 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766060    5503 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766062    5503 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766064    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766066    5503 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766069    5503 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766071    5503 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766074    5503 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766076    5503 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766078    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766081    5503 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766083    5503 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766086    5503 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766088    5503 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:17.771787 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766091    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766093    5503 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766096    5503 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766099    5503 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766101    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766103    5503 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766189    5503 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766196    5503 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766202    5503 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766206    5503 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766211    5503 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766214    5503 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766218    5503 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766222    5503 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766225    5503 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766228    5503 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766231    5503 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766234    5503 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766237    5503 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766240    5503 flags.go:64] FLAG: --cgroup-root=""
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766243    5503 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766246    5503 flags.go:64] FLAG: --client-ca-file=""
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766249    5503 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 22:19:17.772362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766251    5503 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766256    5503 flags.go:64] FLAG: --cluster-domain=""
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766259    5503 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766262    5503 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766265    5503 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766268    5503 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766272    5503 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766275    5503 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766278    5503 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766281    5503 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766284    5503 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766287    5503 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766290    5503 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766293    5503 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766296    5503 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766300    5503 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766303    5503 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766305    5503 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766308    5503 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766311    5503 flags.go:64] FLAG: --enable-server="true"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766314    5503 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766318    5503 flags.go:64] FLAG: --event-burst="100"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766321    5503 flags.go:64] FLAG: --event-qps="50"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766324    5503 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 22:19:17.772958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766327    5503 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766330    5503 flags.go:64] FLAG: --eviction-hard=""
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766333    5503 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766336    5503 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766339    5503 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766342    5503 flags.go:64] FLAG: --eviction-soft=""
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766345    5503 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766347    5503 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766350    5503 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766353    5503 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766356    5503 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766359    5503 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766362    5503 flags.go:64] FLAG: --feature-gates=""
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766366    5503 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766369    5503 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766372    5503 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766375    5503 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766378    5503 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766381    5503 flags.go:64] FLAG: --help="false"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766384    5503 flags.go:64] FLAG: --hostname-override="ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766387    5503 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766389    5503 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766392    5503 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 22:19:17.773563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766395    5503 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766398    5503 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766401    5503 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766403    5503 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766406    5503 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766409    5503 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766412    5503 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766415    5503 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766418    5503 flags.go:64] FLAG: --kube-reserved=""
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766420    5503 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766423    5503 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766426    5503 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766428    5503 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766431    5503 flags.go:64] FLAG: --lock-file=""
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766433    5503 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766436    5503 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766439    5503 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766446    5503 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766448    5503 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766455    5503 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766458    5503 flags.go:64] FLAG: --logging-format="text"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766461    5503 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766464    5503 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766467    5503 flags.go:64] FLAG: --manifest-url=""
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766469    5503 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 22:19:17.774113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766474    5503 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766477    5503 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766481    5503 flags.go:64] FLAG: --max-pods="110"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766484    5503 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766487    5503 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766490    5503 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766492    5503 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766495    5503 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766498    5503 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766500    5503 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766507    5503 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766509    5503 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766512    5503 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766515    5503 flags.go:64] FLAG: --pod-cidr=""
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766518    5503 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766520    5503 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766523    5503 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766526    5503 flags.go:64] FLAG: --port="10250"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766529    5503 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766532    5503 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-021491f51466d18d0"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766535    5503 flags.go:64] FLAG: --qos-reserved=""
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766537    5503 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766541    5503 flags.go:64] FLAG: --register-node="true"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766543    5503 flags.go:64] FLAG: --register-with-taints=""
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766548    5503 flags.go:64] FLAG: --registry-burst="10"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766551    5503 flags.go:64] FLAG: --registry-qps="5"
Oct 30 22:19:17.774693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766555    5503 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766558    5503 flags.go:64] FLAG: --reserved-memory=""
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766561    5503 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766564    5503 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766567    5503 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766569    5503 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766572    5503 flags.go:64] FLAG: --runonce="false"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766575    5503 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766578    5503 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766581    5503 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766584    5503 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766587    5503 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766590    5503 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766592    5503 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766595    5503 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766598    5503 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766601    5503 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766603    5503 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766606    5503 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766609    5503 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766611    5503 flags.go:64] FLAG: --system-cgroups=""
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766614    5503 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766619    5503 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766622    5503 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766625    5503 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 22:19:17.775428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766628    5503 flags.go:64] FLAG: --tls-min-version=""
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766631    5503 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766634    5503 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766636    5503 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766639    5503 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766643    5503 flags.go:64] FLAG: --v="2"
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766647    5503 flags.go:64] FLAG: --version="false"
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766651    5503 flags.go:64] FLAG: --vmodule=""
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766655    5503 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.766658    5503 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766833    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766838    5503 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766842    5503 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766845    5503 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766847    5503 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766851    5503 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766861    5503 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766865    5503 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766868    5503 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766871    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766874    5503 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:17.776025 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766877    5503 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766879    5503 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766882    5503 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766884    5503 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766887    5503 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766889    5503 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766892    5503 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766894    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766897    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766899    5503 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766902    5503 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766904    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766906    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766909    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766911    5503 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766913    5503 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766917    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766919    5503 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766924    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766926    5503 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:17.776607 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766928    5503 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766933    5503 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766936    5503 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766939    5503 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766941    5503 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766944    5503 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766947    5503 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766949    5503 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766957    5503 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766960    5503 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766962    5503 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766965    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766967    5503 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766970    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766972    5503 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766975    5503 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766977    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766979    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766982    5503 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766984    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:17.777254 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766987    5503 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766989    5503 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766992    5503 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766994    5503 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766996    5503 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.766999    5503 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767001    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767003    5503 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767007    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767009    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767013    5503 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767015    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767017    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767020    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767022    5503 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767025    5503 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767028    5503 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767030    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767033    5503 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767035    5503 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:17.778040 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767038    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767046    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767048    5503 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767051    5503 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767053    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767056    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767058    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767061    5503 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767063    5503 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767065    5503 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767068    5503 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767070    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767073    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767075    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767078    5503 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767080    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767082    5503 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767085    5503 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767087    5503 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767089    5503 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:17.778778 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767093    5503 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767096    5503 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767099    5503 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767102    5503 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767104    5503 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767107    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767109    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767112    5503 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767114    5503 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767116    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767119    5503 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767121    5503 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767124    5503 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767127    5503 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767134    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.767160    5503 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:19:17.779495 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767202    5503 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767206    5503 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767209    5503 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767213    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767215    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767218    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767221    5503 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767223    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767226    5503 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767228    5503 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767230    5503 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767233    5503 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767235    5503 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767238    5503 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767246    5503 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767248    5503 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767252    5503 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767254    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767257    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:17.780117 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767260    5503 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767263    5503 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767265    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767268    5503 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767270    5503 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767273    5503 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767275    5503 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767278    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767280    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767282    5503 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767285    5503 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767287    5503 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767297    5503 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767301    5503 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767305    5503 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767308    5503 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767311    5503 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767313    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767316    5503 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:17.780812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767319    5503 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767321    5503 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767323    5503 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767326    5503 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767328    5503 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767331    5503 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767333    5503 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767335    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767338    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767340    5503 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767344    5503 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767346    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767349    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767353    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767355    5503 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767357    5503 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767360    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767362    5503 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767365    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767367    5503 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:17.781506 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767370    5503 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767373    5503 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767376    5503 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767379    5503 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767382    5503 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767385    5503 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767393    5503 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767395    5503 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767398    5503 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767400    5503 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767403    5503 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767405    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767407    5503 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767410    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767412    5503 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767415    5503 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767417    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767419    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767422    5503 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767424    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:17.782041 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767426    5503 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767429    5503 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767432    5503 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767434    5503 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767437    5503 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767440    5503 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767442    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767445    5503 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767447    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767450    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767452    5503 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767454    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767457    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767459    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767462    5503 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767465    5503 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767467    5503 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767470    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767472    5503 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767481    5503 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:17.782493 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767484    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767486    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767489    5503 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767491    5503 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767493    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767496    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767498    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.767501    5503 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.772498    5503 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.772515    5503 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.772537    5503 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.772542    5503 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772600    5503 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772605    5503 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772608    5503 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772611    5503 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772613    5503 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772617    5503 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772619    5503 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:17.782965 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772622    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772625    5503 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772627    5503 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772630    5503 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772632    5503 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772635    5503 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772638    5503 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772640    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772644    5503 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772647    5503 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772650    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772652    5503 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772655    5503 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772657    5503 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772660    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772675    5503 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772678    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772681    5503 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772684    5503 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:17.783434 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772687    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772689    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772692    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772695    5503 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772698    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772700    5503 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772703    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772705    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772708    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772711    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772713    5503 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772716    5503 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772718    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772720    5503 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772723    5503 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772726    5503 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772728    5503 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772731    5503 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772733    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772735    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:17.784144 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772738    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772741    5503 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772744    5503 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772746    5503 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772748    5503 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772751    5503 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772753    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772756    5503 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772758    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772762    5503 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772765    5503 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772767    5503 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772769    5503 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772772    5503 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772774    5503 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772777    5503 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772780    5503 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772782    5503 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772784    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772787    5503 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:17.784640 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772789    5503 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772792    5503 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772794    5503 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772796    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772799    5503 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772801    5503 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772804    5503 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772807    5503 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772811    5503 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772814    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772817    5503 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772819    5503 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772822    5503 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772825    5503 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772828    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772831    5503 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772833    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772836    5503 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772838    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:17.785154 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772841    5503 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772843    5503 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772846    5503 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772849    5503 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772851    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772853    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772856    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772859    5503 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772861    5503 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772864    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772866    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772869    5503 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772872    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772874    5503 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772876    5503 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772879    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772881    5503 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772883    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772886    5503 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772889    5503 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:17.785596 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.772891    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.772918    5503 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773055    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773059    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773062    5503 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773064    5503 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773067    5503 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773070    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773073    5503 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773075    5503 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773077    5503 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773080    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773086    5503 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773091    5503 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773094    5503 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:17.786185 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773096    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773099    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773101    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773104    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773106    5503 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773108    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773111    5503 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773113    5503 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773116    5503 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773118    5503 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773120    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773123    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773125    5503 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773127    5503 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773130    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773132    5503 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773135    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773137    5503 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773140    5503 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:17.786598 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773142    5503 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773144    5503 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773147    5503 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773150    5503 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773153    5503 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773158    5503 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773163    5503 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773165    5503 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773168    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773170    5503 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773172    5503 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773175    5503 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773177    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773180    5503 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773182    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773184    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773186    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773189    5503 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773192    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773194    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:17.787340 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773196    5503 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773200    5503 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773203    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773206    5503 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773208    5503 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773211    5503 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773213    5503 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773215    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773218    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773221    5503 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773223    5503 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773225    5503 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773228    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773230    5503 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773233    5503 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773235    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773237    5503 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773239    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773242    5503 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:17.787953 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773244    5503 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773247    5503 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773249    5503 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773252    5503 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773254    5503 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773257    5503 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773259    5503 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773261    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773263    5503 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773266    5503 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773268    5503 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773270    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773272    5503 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773275    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773277    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773280    5503 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773282    5503 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773284    5503 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773287    5503 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773289    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:17.788583 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773291    5503 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773294    5503 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773296    5503 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773298    5503 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773301    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773303    5503 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773306    5503 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773308    5503 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773311    5503 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773313    5503 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773315    5503 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773318    5503 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773320    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773323    5503 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773325    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.773347    5503 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:19:17.789362 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773382    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773386    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773388    5503 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773392    5503 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773395    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773399    5503 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773402    5503 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773404    5503 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773407    5503 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773409    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773414    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773418    5503 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773422    5503 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773426    5503 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773431    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773434    5503 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773438    5503 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773442    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773446    5503 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:17.789898 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773449    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773452    5503 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773454    5503 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773456    5503 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773459    5503 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773461    5503 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773464    5503 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773466    5503 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773468    5503 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773471    5503 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773473    5503 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773476    5503 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773479    5503 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773481    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773483    5503 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773486    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773488    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773491    5503 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773493    5503 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773495    5503 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:17.790396 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773498    5503 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773500    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773502    5503 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773505    5503 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773507    5503 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773509    5503 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773511    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773514    5503 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773516    5503 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773519    5503 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773521    5503 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773523    5503 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773526    5503 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773528    5503 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773531    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773533    5503 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773535    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773538    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773540    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773542    5503 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:17.790874 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773545    5503 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773547    5503 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773550    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773552    5503 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773555    5503 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773557    5503 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773560    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773564    5503 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773568    5503 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773571    5503 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773573    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773576    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773578    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773581    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773583    5503 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773585    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773588    5503 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773590    5503 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773593    5503 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:17.791359 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773595    5503 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773597    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773600    5503 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773602    5503 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773604    5503 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773607    5503 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773609    5503 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773612    5503 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773614    5503 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773616    5503 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773619    5503 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773621    5503 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773623    5503 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773626    5503 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773628    5503 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773631    5503 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773633    5503 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773635    5503 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773638    5503 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773641    5503 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:17.791812 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773643    5503 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773645    5503 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773648    5503 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773650    5503 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773652    5503 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773654    5503 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773657    5503 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:17.773679    5503 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.773750    5503 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.774641    5503 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.774711    5503 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.774882    5503 server.go:1014] "Starting client certificate rotation"
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.775076    5503 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.775140    5503 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:13:05 +0000 UTC" deadline="2028-07-24 20:09:14.745532318 +0000 UTC"
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.775208    5503 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15189h49m56.970327361s"
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.775320    5503 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:19:17.792420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.776366    5503 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:19:17.792832 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.780752    5503 log.go:25] "Validated CRI v1 runtime API"
Oct 30 22:19:17.792832 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.782953    5503 log.go:25] "Validated CRI v1 image API"
Oct 30 22:19:17.792832 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.783718    5503 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 22:19:17.792832 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.786205    5503 fs.go:135] Filesystem UUIDs: map[4d0947a5-0271-4ced-b80d-f445e80bdd86:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a97a27a4-4e49-4a88-a76a-719ff70c57c6:/dev/nvme0n1p4]
Oct 30 22:19:17.792952 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.786224    5503 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0eb3434f123f1a6456f8503a2c7b791fc092551e23f2b2419be529aa78b77868/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0eb3434f123f1a6456f8503a2c7b791fc092551e23f2b2419be529aa78b77868/userdata/shm major:0 minor:322 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1267cd9cb859f3320e7b5ef1ff38b43344b6f35a52ce78ea2d86b48dfd541210/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1267cd9cb859f3320e7b5ef1ff38b43344b6f35a52ce78ea2d86b48dfd541210/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1594ec48944c128dd92eb89f8bf470a5da5a0419b05610f83939c3bd6dcdad58/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1594ec48944c128dd92eb89f8bf470a5da5a0419b05610f83939c3bd6dcdad58/userdata/shm major:0 minor:307 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/184da35d5d7690b39f4735a6c51c7b9a264784a04e691d74fe0a0733a5b6a684/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/184da35d5d7690b39f4735a6c51c7b9a264784a04e691d74fe0a0733a5b6a684/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1a879aa3adc402f5517ef9cc23434f9de1f238b5ee9e9d36d908604df3c0665e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1a879aa3adc402f5517ef9cc23434f9de1f238b5ee9e9d36d908604df3c0665e/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/27165cb81007e61d7bd6e5e4707adcb2476998b09dac844ae734284963fe1c83/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/27165cb81007e61d7bd6e5e4707adcb2476998b09dac844ae734284963fe1c83/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/293cd367bcc86644f94637fd17e1da0102179470fbeb530dd5434e822ef5683b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/293cd367bcc86644f94637fd17e1da0102179470fbeb530dd5434e822ef5683b/userdata/shm major:0 minor:93 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/376987537f0da72e3f9ddeb28370679e49440d37b4ba82e54697db5b8ff7981a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/376987537f0da72e3f9ddeb28370679e49440d37b4ba82e54697db5b8ff7981a/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3d27ea8736c54565ff75d36add2286e1cfaa9493c0724ae73dee3984bd75e9cd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3d27ea8736c54565ff75d36add2286e1cfaa9493c0724ae73dee3984bd75e9cd/userdata/shm major:0 minor:73 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3e301a910925891def07f423a9e9350fc65d9dc0b259d5d54e710fa2474869b7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3e301a910925891def07f423a9e9350fc65d9dc0b259d5d54e710fa2474869b7/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/533726c955f944a49a6efe13c0d836e8f1bf9689485cc747ea2b65cb09ef8962/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/533726c955f944a49a6efe13c0d836e8f1bf9689485cc747ea2b65cb09ef8962/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/625923e6d4b4c44813dd2cdcf534575a058bc3fcf5722e0de8c33305b7394594/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/625923e6d4b4c44813dd2cdcf534575a058bc3fcf5722e0de8c33305b7394594/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6773a6575e687d07c3cfc972a450eb73e02320c5ef2c3d3b67ae40dd53805e9c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6773a6575e687d07c3cfc972a450eb73e02320c5ef2c3d3b67ae40dd53805e9c/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/67fa66cc090e754099433c576545e02bd6b4f736fa6c2df10b46f01eb9e9ed35/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/67fa66cc090e754099433c576545e02bd6b4f736fa6c2df10b46f01eb9e9ed35/userdata/shm major:0 minor:384 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6feaa293f879d3744c71c3a5cd4fbabb405776faf2c1b3a13198227247b500ad/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6feaa293f879d3744c71c3a5cd4fbabb405776faf2c1b3a13198227247b500ad/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/836930fd1c99878552b3a1bbb4dc049f0c9bc41449fa23d5df4bb6c3b8b8d9bb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/836930fd1c99878552b3a1bbb4dc049f0c9bc41449fa23d5df4bb6c3b8b8d9bb/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/893b03549c7592aedb81894711720581ed8582c228f7c92211071fcf98618961/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/893b03549c7592aedb81894711720581ed8582c228f7c92211071fcf98618961/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/933e5d368b5142c4f53b2b2438a6fe6ae1aae2936b0cfc2e599f5ea00e709727/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/933e5d368b5142c4f53b2b2438a6fe6ae1aae2936b0cfc2e599f5ea00e709727/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aa75e6408b852ec372edc27d5e83df8542d3250cdfc227111b442e043141b976/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aa75e6408b852ec372edc27d5e83df8542d3250cdfc227111b442e043141b976/userdata/shm major:0 minor:331 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b1abc849912eaceb78025dbe82def1660754cf121c3aa234f6343ee0e9f10148/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b1abc849912eaceb78025dbe82def1660754cf121c3aa234f6343ee0e9f10148/userdata/shm major:0 minor:383 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c9241f3113201235de7daaefcfde220c853f7de35113b0df014bdf59d39e88af/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c9241f3113201235de7daaefcfde220c853f7de35113b0df014bdf59d39e88af/userdata/shm major:0 minor:381 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dc1164eca5be445f995c88bd2c858c3155e577684f706032dbf418a2c09e103d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dc1164eca5be445f995c88bd2c858c3155e577684f706032dbf418a2c09e103d/userdata/shm major:0 minor:282 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff634edbd2c1ec04cba1038ff5f6834e80222d86bcdecc21de57862cbb681415/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff634edbd2c1ec04cba1038ff5f6834e80222d86bcdecc21de57862cbb681415/userdata/shm major:0 minor:396 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79/volumes/kubernetes.io~projected/kube-api-access-hmhgd:{mountpoint:/var/lib/kubelet/pods/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79/volumes/kubernetes.io~projected/kube-api-access-hmhgd major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1ae3621a-b431-4675-8617-37461922d241/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/1ae3621a-b431-4675-8617-37461922d241/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:375 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21192d3f-1a3f-4a32-ab4a-70fbcada9424/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/21192d3f-1a3f-4a32-ab4a-70fbcada9424/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21192d3f-1a3f-4a32-ab4a-70fbcada9424/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/21192d3f-1a3f-4a32-ab4a-70fbcada9424/volumes/kubernetes.io~empty-dir/tmp major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21192d3f-1a3f-4a32-ab4a-70fbcada9424/volumes/kubernetes.io~projected/kube-api-access-668ht:{mountpoint:/var/lib/kubelet/pods/21192d3f-1a3f-4a32-ab4a-70fbcada9424/volumes/kubernetes.io~projected/kube-api-access-668ht major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/volumes/kubernetes.io~projected/kube-api-access-sdx5q:{mountpoint:/var/lib/kubelet/pods/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/volumes/kubernetes.io~projected/kube-api-access-sdx5q major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/376fe364-ccc4-4435-b5e0-bef1e6715b26/volumes/kubernetes.io~projected/kube-api-access-86xfb:{mountpoint:/var/lib/kubelet/pods/376fe364-ccc4-4435-b5e0-bef1e6715b26/volumes/kubernetes.io~projected/kube-api-access-86xfb major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/376fe364-ccc4-4435-b5e0-bef1e6715b26/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/376fe364-ccc4-4435-b5e0-bef1e6715b26/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes/kubernetes.io~projected/kube-api-access-48zdx:{mountpoint:/var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes/kubernetes.io~projected/kube-api-access-48zdx major:0 minor:395 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:393 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:394 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4e972ba1-2295-49d0-998b-00b86a1737af/volumes/kubernetes.io~projected/kube-api-access-5wwpn:{mountpoint:/var/lib/kubelet/pods/4e972ba1-2295-49d0-998b-00b86a1737af/volumes/kubernetes.io~projected/kube-api-access-5wwpn major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4e972ba1-2295-49d0-998b-00b86a1737af/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/4e972ba1-2295-49d0-998b-00b86a1737af/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5f9fcff5-40fb-422e-9319-50a1c4156131/volumes/kubernetes.io~projected/kube-api-access-npk9p:{mountpoint:/var/lib/kubelet/pods/5f9fcff5-40fb-422e-9319-50a1c4156131/volumes/kubernetes.io~projected/kube-api-access-npk9p major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3/volumes/kubernetes.io~projected/kube-api-access-xgg87:{mountpoint:/var/lib/kubelet/pods/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3/volumes/kubernetes.io~projected/kube-api-access-xgg87 major:0 minor:380 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/84f3a1bb-9373-443e-b957-1bf6ccb03477/volumes/kubernetes.io~projected/kube-api-access-gm9gc:{mountpoint:/var/lib/kubelet/pods/84f3a1bb-9373-443e-b957-1bf6ccb03477/volumes/kubernetes.io~projected/kube-api-access-gm9gc major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~projected/kube-api-access-xtppr:{mountpoint:/var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~projected/kube-api-access-xtppr major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9115c245-4790-4124-a435-24b667fc1744/volumes/kubernetes.io~projected/kube-api-access-zq6r9:{mountpoint:/var/lib/kubelet/pods/9115c245-4790-4124-a435-24b667fc1744/volumes/kubernetes.io~projected/kube-api-access-zq6r9 major:0 minor:321 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9115c245-4790-4124-a435-24b667fc1744/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/9115c245-4790-4124-a435-24b667fc1744/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:320 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~projected/kube-api-access-rmk4b:{mountpoint:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~projected/kube-api-access-rmk4b major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a410d5a1-3091-405c-a1b2-9d790b77b458/volumes/kubernetes.io~projected/kube-api-access-pwgtv:{mountpoint:/var/lib/kubelet/pods/a410d5a1-3091-405c-a1b2-9d790b77b458/volumes/kubernetes.io~projected/kube-api-access-pwgtv major:0 minor:243 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a410d5a1-3091-405c-a1b2-9d790b77b458/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/a410d5a1-3091-405c-a1b2-9d790b77b458/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:242 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a410d5a1-3091-405c-a1b2-9d790b77b458/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/a410d5a1-3091-405c-a1b2-9d790b77b458/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:241 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~projected/kube-api-access-89prq:{mountpoint:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~projected/kube-api-access-89prq major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:275 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd11fe2e-49c0-4478-928a-a04ed1f29273/volumes/kubernetes.io~projected/kube-api-access-wcj9x:{mountpoint:/var/lib/kubelet/pods/cd11fe2e-49c0-4478-928a-a04ed1f29273/volumes/kubernetes.io~projected/kube-api-access-wcj9x major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd11fe2e-49c0-4478-928a-a04ed1f29273/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/cd11fe2e-49c0-4478-928a-a04ed1f29273/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d967b5ff-61dc-4c9a-8705-d735e0075db5/volumes/kubernetes.io~projected/kube-api-access-nltc9:{mountpoint:/var/lib/kubelet/pods/d967b5ff-61dc-4c9a-8705-d735e0075db5/volumes/kubernetes.io~projected/kube-api-access-nltc9 major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1/volumes/kubernetes.io~projected/kube-api-access-7xdjz:{mountpoint:/var/lib/kubelet/pods/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1/volumes/kubernetes.io~projected/kube-api-access-7xdjz major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1/volumes/kubernetes.io~secret/metrics-certs major:0 minor:379 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f0f05f12-85f1-4c1d-9115-e1cd2a336b55/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/f0f05f12-85f1-4c1d-9115-e1cd2a336b55/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f3ef0c70-9090-44d0-bb59-95ab1cf0c245/volumes/kubernetes.io~projected/kube-api-access-ph67d:{mountpoint:/var/lib/kubelet/pods/f3ef0c70-9090-44d0-bb59-95ab1cf0c245/volumes/kubernetes.io~projected/kube-api-access-ph67d major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6f0136b-88a1-427c-92bc-5cc7ed9d7030/volumes/kubernetes.io~projected/kube-api-access-64j65:{mountpoint:/var/lib/kubelet/pods/f6f0136b-88a1-427c-92bc-5cc7ed9d7030/volumes/kubernetes.io~projected/kube-api-access-64j65 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb7545c5-92b9-4107-93d0-f0617277df0f/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/fb7545c5-92b9-4107-93d0-f0617277df0f/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb7545c5-92b9-4107-93d0-f0617277df0f/volumes/kubernetes.io~projected/kube-api-access-4cb74:{mountpoint:/var/lib/kubelet/pods/fb7545c5-92b9-4107-93d0-f0617277df0f/volumes/kubernetes.io~projected/kube-api-access-4cb74 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb7545c5-92b9-4107-93d0-f0617277df0f/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/fb7545c5-92b9-4107-93d0-f0617277df0f/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:58 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/8bd152301dd6e824336702cd25fcaf92c8b6a7713274df071628b22d5da0d8b8/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/31f6a9e2bbef8898a3e68878ff026b36efc49a2261caf175d38d4f8432482967/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/5fca176fe80c5c7dd85e847ce0869631c9828bb8b74e4a9b4c9f4d99bfbad20e/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/3773a3113e5b49e5568961c8d36a0860fec5f2be5a6aedb54775e0face3885a3/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/1e480a6aede54cf8486542b969c1016271f661585ec3cc318536d50d8ec27403/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/23249c4e50e41c2e8d2cc0569b956113767cd25821c0a18c1fa03eb5545a8513/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/3c6d8d158611bfe571de2f627dff97b7f3277f848be8eca483e908041a16aace/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/8842de6ff24b16548904e2c3e91372ea9cdc4560aeca80fdba2a88b3a77da52e/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/355be83889b098682e1f8ff85f92e4a5975696e0928d8769c0383609cb4e9f6b/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/006f9973118a20ad79c8b864990b8cbfa10301f666abbcd8a86ae66944e5d6a4/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/bf3e1ce1f4cb7d003aa996ac05d204aa83fe8630b0a44a8c978938c89e363eb9/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-145:{mountpoint:/var/lib/containers/storage/overlay/3ecbdb7ab9b7f1a587ac318f0c77f13953f6539902862d398116cb93e7d7a97b/merged major:0 minor:145 fsType:overlay blockSize:0} overlay_0-148:{mountpoint:/var/lib/containers/storage/overlay/dfdd32a060b6d64346ef7ae654f4d1bb2410dda2f47a3623600f4a0ce7169781/merged major:0 minor:148 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/2f92886afde84b0ec1ffe4b949db2efa18bd77d91cf7d700bf2c9d85c2895970/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/29225f56d4d48f3799e65475cc22b9262787cfe6d2c21d5d309530f41c176839/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/5bce504d0829c9fd66a3ec521f1f00413b9687d27495d245c9fd5263dc7d666c/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/c39ecbfb936def858f23f1735bf3aceb65c0b549f6182b60a6eff5472ecb4c7a/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/aec550798ea7cfa629322cb0f5d6f5120d8ad48ded0c0e87c73ae7df0fa46d3f/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/71b4926fb0edaf70d0e8748966f8a5743954cbf121debb302fe99d5d2e57ac01/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/c63beaea64d7f214edf8dfac0948efa9402d0fa3459ec581fce9914c3c96bbaa/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/68b910c283f0df66b138f9dc0ce5e8ff374c3549d4ce19bae171ca4d66699272/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/b43fcc8aeb62491194d4c5d9107f293c2fd8ab1dda0e4b0e44ecfe96f740b754/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/880e1110954406e37c2ab50bcf83cd1546ef5b911e7ae12937a5edf7571a29ce/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/932e2e9efb8465b4fcecc9815e187fa3a972b12796345dd3735bd539f5e047a3/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/c7a80ba761be5b5256ac98e10223b91998bac281e61f55549a6552c0cecb6dec/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/4bf7f8f97b9e63bd2db3c823ef810daee0d8a9c7f62fcef3d379413cfc08689c/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/001cf16463a2dc917afbd74365a057541909a7c750f178cce46095eee68fedc0/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/87c9165f16a7de77171188a797586e320c5424ffc6c2988c0f75d15e44b660f4/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/4efd70d32f27aee4ea0ee668fc7e835496e1a3b433f2825484f4e7fef4de5050/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-249:{mountpoint:/var/lib/containers/storage/overlay/f4ff7ff2b86cc0535e994e8f5126694460c292479bc0edf465f8e8edb00aab09/merged major:0 minor:249 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/a7f988a81e3a39a35f73357b16e85fe55ef4bbb92bf22bfb4e9a513b2c24d34c/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-257:{mountpoint:/var/lib/containers/storage/overlay/7f9a2ca53008135d1733558c641956862bb453302d7fd2a5d3a93310e01ca897/merged major:0 minor:257 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/8c303ff7c16f429f04bb41553f4eb5eb60013c39877081943760d0a8d1182695/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/106cba4645cba6965c608238318f26b9c3169e7529b565e34775d03dddbe9c8e/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/775d4599fd4b8a746f8cf3549a83f69f848b892eba8ee8492e8f0b95751bd27a/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/458e519d2b8f62bcba2664f31248afd30b9d39f719960c5b8b58722c8c1ae1d6/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-288:{mountpoint:/var/lib/containers/storage/overlay/4cff9cdcbb79c55a5e2dce5e64323eda7686ae60e2cb7b5385182e81460a5e56/merged major:0 minor:288 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/1292de116f01443e2f5b6715a0d57ce71bdd268caa5f78e955a5594700cd9d2a/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/0d08f996de80369303ab9665fe9fef50d9110ccb6a89acd8510b66fa0237a2e5/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/096864210e95c75ad98e1a0d33048e9d39342245fbdd73bd97796d2b38f5e0f9/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-315:{mountpoint:/var/lib/containers/storage/overlay/6ec0afad8a5ba8c5df02578c8364d479421ad6f065e80ad0217d69ff2a58fc54/merged major:0 minor:315 fsType:overlay blockSize:0} overlay_0-324:{mountpoint:/var/lib/containers/storage/overlay/38d20074bcfdcc73964808f219c9d54402e9257dfed8ca19b7bab680a126b35c/merged major:0 minor:324 fsType:overlay blockSize:0} overlay_0-333:{mountpoint:/var/lib/containers/storage/overlay/61627020cd603e85acf23184c948c269a61d4bdb100faff7a78111a16deaf204/merged major:0 minor:333 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/ed0bea8b577229c071cbb492b26d7c9641d53da279da2e423b3ac859291bc727/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-337:{mountpoint:/var/lib/containers/storage/overlay/68083dc6d2c804249e9f595d577ce8e5f4bdcecf4bf36dcab056807ac2b8f266/merged major:0 minor:337 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/cdf68b3dd1cd50b2005fc7d55e1892fdb02a0a6f9bb41b552c9962d1a0966a23/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-347:{mountpoint:/var/lib/containers/storage/overlay/a53e6bc628b09d9005e0712adef8eaba8c15969633b9d6d36f774f702ad9d2e8/merged major:0 minor:347 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/d1c4f665f1ea3fb73a6debd22971e85bfa7063c2ae7780b8ce8223cb1235cadb/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-358:{mountpoint:/var/lib/containers/storage/overlay/f4f9b1f07cf1199dd10c8d951d69bdd0c629264527ab5f952db494354b224786/merged major:0 minor:358 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/cd82266aca2a6fa249acf268b41cd081708e906e21ecf3fcf95bd1d313c43315/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-370:{mountpoint:/var/lib/containers/storage/overlay/e4fa36042d2877c3a9cd192b0602da35d2e16267727c502443b5a562715323df/merged major:0 minor:370 fsType:overlay blockSize:0} overlay_0-387:{mountpoint:/var/lib/containers/storage/overlay/1814d241f16c5ae571e27746cc75d32fa32e8bd417eaa264e187e8debd050a8b/merged major:0 minor:387 fsType:overlay blockSize:0} overlay_0-389:{mountpoint:/var/lib/containers/storage/overlay/e1f51874b1ace19d522b2fa5634d6d7362f2df41eef7b8b57d79d48797d005dd/merged major:0 minor:389 fsType:overlay blockSize:0} overlay_0-391:{mountpoint:/var/lib/containers/storage/overlay/4592e35f6ff9a6212d4efe7bd7e2873fa29a0cdbe9015f0203019f0e92f612f1/merged major:0 minor:391 fsType:overlay blockSize:0} overlay_0-398:{mountpoint:/var/lib/containers/storage/overlay/b84bdaefad2dd66b806271d6082b5afcd2b5d3d9745e16753ac4e7dab83ed0bb/merged major:0 minor:398 fsType:overlay blockSize:0} overlay_0-400:{mountpoint:/var/lib/containers/storage/overlay/6d29f16ceb5fe54265b2d69ff4d3b945193cf79657ea90b90738ea7981c33b8f/merged major:0 minor:400 fsType:overlay blockSize:0} overlay_0-402:{mountpoint:/var/lib/containers/storage/overlay/501ed9d782d902b8faff474d8416f2546bfecbb7840069e9f9ea461f4ee49b66/merged major:0 minor:402 fsType:overlay blockSize:0} overlay_0-408:{mountpoint:/var/lib/containers/storage/overlay/8599e03417b98f29d8063c16ad978507b6c582211ecc0f19887d861e9022b56d/merged major:0 minor:408 fsType:overlay blockSize:0} overlay_0-413:{mountpoint:/var/lib/containers/storage/overlay/f94c84e87202fa38440f5581ebe6602ec95fdf6780d5fd8bbe857fa580c72da5/merged major:0 minor:413 fsType:overlay blockSize:0} overlay_0-419:{mountpoint:/var/lib/containers/storage/overlay/81117f00f4da3d33bb8062ef9d0c2d9f371f4eb67da7551bd7528b8ed54480be/merged major:0 minor:419 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/9c267494b2b84735d25e9776d407984b0abfd3328089cfe36916b7ccf69c5e91/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/61e7739f70087e83da4e04027badc47858fd364fe432efa2255535bad74670d7/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/58a903b33f3c4ad29b47de0d8a2bd3e88c437ba15200011783e59bbafd097a4e/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/12d2afe5f52dc8b3a9d29b57f0b7a44495c488848bd25d8494abbaee2b12e911/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/29d31ba93ea77f8f744caacf8448d8f3dc73082840ba2bf8dd41d8208ad329f2/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/b0493a7282618071d93c8968d74b0ccf41ea7caa48c23e768ae434212614ae84/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/0b1dfdcc0611fbe0bc089a4062514ee2a2a24acc4281922cdf6f81da6e023598/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/aa8376d8176678d23c19983a73ceb1fdb28c06e9fd4c9d8a52e6810857afac4c/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/a2633b0b99f3ee4eec49238a01cb79a6e4d9dc779c252e84fdecee8fd1c9f2d4/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/c9d6a71d30210e6beab6ec673f72a0c7a7001fd31121c24a47e6e5318aed9744/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 22:19:17.795819 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.795369    5503 manager.go:217] Machine: {Timestamp:2026-10-30 22:19:17.794689628 +0000 UTC m=+0.063771148 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3167370 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25d1ddf396fd3b32c148615834826f SystemUUID:ec25d1dd-f396-fd3b-32c1-48615834826f BootID:4d1d830c-ccaa-403f-aaff-6b03e2f00d5e Filesystems:[{Device:overlay_0-402 DeviceMajor:0 DeviceMinor:402 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f0f05f12-85f1-4c1d-9115-e1cd2a336b55/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3d27ea8736c54565ff75d36add2286e1cfaa9493c0724ae73dee3984bd75e9cd/userdata/shm DeviceMajor:0 DeviceMinor:73 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9115c245-4790-4124-a435-24b667fc1744/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:320 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1a879aa3adc402f5517ef9cc23434f9de1f238b5ee9e9d36d908604df3c0665e/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-347 DeviceMajor:0 DeviceMinor:347 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/fb7545c5-92b9-4107-93d0-f0617277df0f/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f3ef0c70-9090-44d0-bb59-95ab1cf0c245/volumes/kubernetes.io~projected/kube-api-access-ph67d DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4e972ba1-2295-49d0-998b-00b86a1737af/volumes/kubernetes.io~projected/kube-api-access-5wwpn DeviceMajor:0 DeviceMinor:204 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3e301a910925891def07f423a9e9350fc65d9dc0b259d5d54e710fa2474869b7/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cd11fe2e-49c0-4478-928a-a04ed1f29273/volumes/kubernetes.io~projected/kube-api-access-wcj9x DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-398 DeviceMajor:0 DeviceMinor:398 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a410d5a1-3091-405c-a1b2-9d790b77b458/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:242 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/a410d5a1-3091-405c-a1b2-9d790b77b458/volumes/kubernetes.io~projected/kube-api-access-pwgtv DeviceMajor:0 DeviceMinor:243 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/27165cb81007e61d7bd6e5e4707adcb2476998b09dac844ae734284963fe1c83/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aa75e6408b852ec372edc27d5e83df8542d3250cdfc227111b442e043141b976/userdata/shm DeviceMajor:0 DeviceMinor:331 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-358 DeviceMajor:0 DeviceMinor:358 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:379 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-400 DeviceMajor:0 DeviceMinor:400 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-145 DeviceMajor:0 DeviceMinor:145 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/184da35d5d7690b39f4735a6c51c7b9a264784a04e691d74fe0a0733a5b6a684/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dc1164eca5be445f995c88bd2c858c3155e577684f706032dbf418a2c09e103d/userdata/shm DeviceMajor:0 DeviceMinor:282 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-389 DeviceMajor:0 DeviceMinor:389 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/293cd367bcc86644f94637fd17e1da0102179470fbeb530dd5434e822ef5683b/userdata/shm DeviceMajor:0 DeviceMinor:93 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/volumes/kubernetes.io~projected/kube-api-access-sdx5q DeviceMajor:0 DeviceMinor:270 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:275 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff634edbd2c1ec04cba1038ff5f6834e80222d86bcdecc21de57862cbb681415/userdata/shm DeviceMajor:0 DeviceMinor:396 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6feaa293f879d3744c71c3a5cd4fbabb405776faf2c1b3a13198227247b500ad/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/d967b5ff-61dc-4c9a-8705-d735e0075db5/volumes/kubernetes.io~projected/kube-api-access-nltc9 DeviceMajor:0 DeviceMinor:63 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/376fe364-ccc4-4435-b5e0-bef1e6715b26/volumes/kubernetes.io~projected/kube-api-access-86xfb DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~projected/kube-api-access-rmk4b DeviceMajor:0 DeviceMinor:330 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fb7545c5-92b9-4107-93d0-f0617277df0f/volumes/kubernetes.io~projected/kube-api-access-4cb74 DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-324 DeviceMajor:0 DeviceMinor:324 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-387 DeviceMajor:0 DeviceMinor:387 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4e972ba1-2295-49d0-998b-00b86a1737af/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:203 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/376987537f0da72e3f9ddeb28370679e49440d37b4ba82e54697db5b8ff7981a/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:279 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~projected/kube-api-access-89prq DeviceMajor:0 DeviceMinor:281 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:329 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1/volumes/kubernetes.io~projected/kube-api-access-7xdjz DeviceMajor:0 DeviceMinor:66 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-288 DeviceMajor:0 DeviceMinor:288 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/21192d3f-1a3f-4a32-ab4a-70fbcada9424/volumes/kubernetes.io~projected/kube-api-access-668ht DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/84f3a1bb-9373-443e-b957-1bf6ccb03477/volumes/kubernetes.io~projected/kube-api-access-gm9gc DeviceMajor:0 DeviceMinor:64 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/893b03549c7592aedb81894711720581ed8582c228f7c92211071fcf98618961/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cd11fe2e-49c0-4478-928a-a04ed1f29273/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~projected/kube-api-access-xtppr DeviceMajor:0 DeviceMinor:299 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:276 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-315 DeviceMajor:0 DeviceMinor:315 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:327 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6773a6575e687d07c3cfc972a450eb73e02320c5ef2c3d3b67ae40dd53805e9c/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:265 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-337 DeviceMajor:0 DeviceMinor:337 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/533726c955f944a49a6efe13c0d836e8f1bf9689485cc747ea2b65cb09ef8962/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3/volumes/kubernetes.io~projected/kube-api-access-xgg87 DeviceMajor:0 DeviceMinor:380 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/625923e6d4b4c44813dd2cdcf534575a058bc3fcf5722e0de8c33305b7394594/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1267cd9cb859f3320e7b5ef1ff38b43344b6f35a52ce78ea2d86b48dfd541210/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:293 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:297 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0eb3434f123f1a6456f8503a2c7b791fc092551e23f2b2419be529aa78b77868/userdata/shm DeviceMajor:0 DeviceMinor:322 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:328 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f6f0136b-88a1-427c-92bc-5cc7ed9d7030/volumes/kubernetes.io~projected/kube-api-access-64j65 DeviceMajor:0 DeviceMinor:65 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:280 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-370 DeviceMajor:0 DeviceMinor:370 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1ae3621a-b431-4675-8617-37461922d241/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:375 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-391 DeviceMajor:0 DeviceMinor:391 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes/kubernetes.io~projected/kube-api-access-48zdx DeviceMajor:0 DeviceMinor:395 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:277 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-408 DeviceMajor:0 DeviceMinor:408 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5f9fcff5-40fb-422e-9319-50a1c4156131/volumes/kubernetes.io~projected/kube-api-access-npk9p DeviceMajor:0 DeviceMinor:67 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:278 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/67fa66cc090e754099433c576545e02bd6b4f736fa6c2df10b46f01eb9e9ed35/userdata/shm DeviceMajor:0 DeviceMinor:384 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:394 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-148 DeviceMajor:0 DeviceMinor:148 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a410d5a1-3091-405c-a1b2-9d790b77b458/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:241 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/376fe364-ccc4-4435-b5e0-bef1e6715b26/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-257 DeviceMajor:0 DeviceMinor:257 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79/volumes/kubernetes.io~projected/kube-api-access-hmhgd DeviceMajor:0 DeviceMinor:62 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/836930fd1c99878552b3a1bbb4dc049f0c9bc41449fa23d5df4bb6c3b8b8d9bb/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9701f603-d98d-49b5-adc7-3550bdab41d2/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:326 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/933e5d368b5142c4f53b2b2438a6fe6ae1aae2936b0cfc2e599f5ea00e709727/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-249 DeviceMajor:0 DeviceMinor:249 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8db49c21-a7b4-4fc6-8342-95243844e22f/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:298 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c9241f3113201235de7daaefcfde220c853f7de35113b0df014bdf59d39e88af/userdata/shm DeviceMajor:0 DeviceMinor:381 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:393 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb7545c5-92b9-4107-93d0-f0617277df0f/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:269 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9115c245-4790-4124-a435-24b667fc1744/volumes/kubernetes.io~projected/kube-api-access-zq6r9 DeviceMajor:0 DeviceMinor:321 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-333 DeviceMajor:0 DeviceMinor:333 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/21192d3f-1a3f-4a32-ab4a-70fbcada9424/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b1abc849912eaceb78025dbe82def1660754cf121c3aa234f6343ee0e9f10148/userdata/shm DeviceMajor:0 DeviceMinor:383 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-413 DeviceMajor:0 DeviceMinor:413 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-419 DeviceMajor:0 DeviceMinor:419 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/21192d3f-1a3f-4a32-ab4a-70fbcada9424/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1594ec48944c128dd92eb89f8bf470a5da5a0419b05610f83939c3bd6dcdad58/userdata/shm DeviceMajor:0 DeviceMinor:307 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0eb3434f123f1a6 MacAddress:4e:d7:5e:e6:97:48 Speed:10000 Mtu:8901} {Name:1267cd9cb859f33 MacAddress:ba:90:aa:48:6d:9f Speed:10000 Mtu:8901} {Name:1594ec48944c128 MacAddress:02:ef:0d:ce:96:b5 Speed:10000 Mtu:8901} {Name:376987537f0da72 MacAddress:9e:e3:6f:54:08:3b Speed:10000 Mtu:8901} {Name:3e301a910925891 MacAddress:c6:d5:9f:f2:e3:58 Speed:10000 Mtu:8901} {Name:533726c955f944a MacAddress:be:b0:5a:0b:e2:bb Speed:10000 Mtu:8901} {Name:67fa66cc090e754 MacAddress:96:ca:7f:cb:34:78 Speed:10000 Mtu:8901} {Name:aa75e6408b852ec MacAddress:f2:b8:ee:fb:27:4a Speed:10000 Mtu:8901} {Name:b1abc849912eace MacAddress:f2:31:9b:bb:ea:1a Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f3:9a:88:11 Speed:0 Mtu:9001} {Name:br-int MacAddress:da:88:ed:4e:62:33 Speed:0 Mtu:8901} {Name:c9241f311320123 MacAddress:22:02:b9:4b:91:b8 Speed:10000 Mtu:8901} {Name:dc1164eca5be445 MacAddress:72:fd:d8:13:39:ba Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f3:9a:88:11 Speed:0 Mtu:9001} {Name:ff634edbd2c1ec0 MacAddress:7a:c6:53:78:d0:63 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:8a:45:74:cc:aa:bb Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:76:2e:09:17:99:1b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 22:19:17.795819 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.795824    5503 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 22:19:17.796150 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.795869    5503 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 22:19:17.796250 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796220    5503 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 22:19:17.796408 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796252    5503 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-97.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 22:19:17.796483 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796411    5503 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 22:19:17.796483 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796423    5503 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 22:19:17.796483 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796440    5503 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:19:17.796483 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796467    5503 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:19:17.796639 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796588    5503 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 22:19:17.796818 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796804    5503 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 22:19:17.796938 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796926    5503 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 22:19:17.796990 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796973    5503 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 22:19:17.796990 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.796986    5503 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 22:19:17.797072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.797001    5503 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 22:19:17.797072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.797015    5503 kubelet.go:405] "Adding apiserver pod source"
Oct 30 22:19:17.797072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.797025    5503 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 22:19:17.797712 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.797696    5503 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 22:19:17.797712 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.797711    5503 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:19:17.798229 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.798211    5503 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 22:19:17.800623 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.800609    5503 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 22:19:17.800623 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.800622    5503 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:19:17.800733 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.800725    5503 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 22:19:17.800900 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.800890    5503 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 22:19:17.800931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.800918    5503 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 22:19:17.801024 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801012    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 22:19:17.801072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801032    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 22:19:17.801072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801040    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 22:19:17.801072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801047    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 22:19:17.801072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801054    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 22:19:17.801072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801061    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 22:19:17.801072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801069    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 22:19:17.801072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801073    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 22:19:17.801343 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801080    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 22:19:17.801343 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801086    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 22:19:17.801343 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801093    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 22:19:17.801343 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801103    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 22:19:17.801343 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801127    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 22:19:17.801343 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.801132    5503 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 22:19:17.805811 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.804941    5503 server.go:1263] "Started kubelet"
Oct 30 22:19:17.805422 ip-10-0-128-97 systemd[1]: Started Kubernetes Kubelet.
Oct 30 22:19:17.806360 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.806124    5503 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 22:19:17.806360 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.806105    5503 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 22:19:17.806360 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.806190    5503 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 22:19:17.806854 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.806786    5503 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 22:19:17.807869 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.807841    5503 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 22:19:17.810755 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.810731    5503 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 22:19:17.810862 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.810839    5503 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 22:19:17.810862 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.810852    5503 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 22:19:17.810993 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:19:17.810887    5503 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-128-97.us-east-2.compute.internal\" not found"
Oct 30 22:19:17.810993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.810937    5503 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.812349    5503 factory.go:55] Registering systemd factory
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.812369    5503 factory.go:223] Registration of the systemd container factory successfully
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.812449    5503 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.812593    5503 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:13:07 +0000 UTC" deadline="2028-07-18 16:11:51.105407324 +0000 UTC"
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.812630    5503 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15041h52m33.292781299s"
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813732    5503 factory.go:153] Registering CRI-O factory
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813745    5503 factory.go:223] Registration of the crio container factory successfully
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813796    5503 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813816    5503 factory.go:103] Registering Raw factory
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813831    5503 manager.go:1199] Started watching for new ooms in manager
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813829    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb7545c5-92b9-4107-93d0-f0617277df0f" volumeName="kubernetes.io/secret/fb7545c5-92b9-4107-93d0-f0617277df0f-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813875    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1ae3621a-b431-4675-8617-37461922d241" volumeName="kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813892    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2" volumeName="kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813907    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bae499a-6abf-4f3a-8e0b-9bb821884f74" volumeName="kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813927    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e972ba1-2295-49d0-998b-00b86a1737af" volumeName="kubernetes.io/configmap/4e972ba1-2295-49d0-998b-00b86a1737af-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813944    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9115c245-4790-4124-a435-24b667fc1744" volumeName="kubernetes.io/projected/9115c245-4790-4124-a435-24b667fc1744-kube-api-access-zq6r9" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813959    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b6fa19a-6fb7-40b6-9073-0a52cfca6b79" volumeName="kubernetes.io/projected/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-kube-api-access-hmhgd" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813977    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f9fcff5-40fb-422e-9319-50a1c4156131" volumeName="kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.813991    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="84f3a1bb-9373-443e-b957-1bf6ccb03477" volumeName="kubernetes.io/projected/84f3a1bb-9373-443e-b957-1bf6ccb03477-kube-api-access-gm9gc" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814002    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8db49c21-a7b4-4fc6-8342-95243844e22f" volumeName="kubernetes.io/empty-dir/8db49c21-a7b4-4fc6-8342-95243844e22f-audit-log" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814016    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9701f603-d98d-49b5-adc7-3550bdab41d2" volumeName="kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-client-tls" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814028    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a410d5a1-3091-405c-a1b2-9d790b77b458" volumeName="kubernetes.io/projected/a410d5a1-3091-405c-a1b2-9d790b77b458-kube-api-access-pwgtv" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814041    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="376fe364-ccc4-4435-b5e0-bef1e6715b26" volumeName="kubernetes.io/projected/376fe364-ccc4-4435-b5e0-bef1e6715b26-kube-api-access-86xfb" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814053    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bae499a-6abf-4f3a-8e0b-9bb821884f74" volumeName="kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814067    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bae499a-6abf-4f3a-8e0b-9bb821884f74" volumeName="kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814078    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a410d5a1-3091-405c-a1b2-9d790b77b458" volumeName="kubernetes.io/configmap/a410d5a1-3091-405c-a1b2-9d790b77b458-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814091    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f" volumeName="kubernetes.io/projected/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-kube-api-access-89prq" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814101    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1" volumeName="kubernetes.io/projected/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-kube-api-access-7xdjz" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814111    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0f05f12-85f1-4c1d-9115-e1cd2a336b55" volumeName="kubernetes.io/secret/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-agent-certs" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814123    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b6fa19a-6fb7-40b6-9073-0a52cfca6b79" volumeName="kubernetes.io/empty-dir/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-tmp-dir" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814136    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2" volumeName="kubernetes.io/empty-dir/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-textfile" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814148    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0f05f12-85f1-4c1d-9115-e1cd2a336b55" volumeName="kubernetes.io/configmap/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-konnectivity-ca" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814180    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21192d3f-1a3f-4a32-ab4a-70fbcada9424" volumeName="kubernetes.io/projected/21192d3f-1a3f-4a32-ab4a-70fbcada9424-kube-api-access-668ht" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814193    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2" volumeName="kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814204    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bae499a-6abf-4f3a-8e0b-9bb821884f74" volumeName="kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca" seLinuxMountContext=""
Oct 30 22:19:17.814196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814215    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f9fcff5-40fb-422e-9319-50a1c4156131" volumeName="kubernetes.io/projected/5f9fcff5-40fb-422e-9319-50a1c4156131-kube-api-access-npk9p" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814225    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9701f603-d98d-49b5-adc7-3550bdab41d2" volumeName="kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814261    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a410d5a1-3091-405c-a1b2-9d790b77b458" volumeName="kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814268    5503 manager.go:319] Starting recovery of all containers
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814279    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bae499a-6abf-4f3a-8e0b-9bb821884f74" volumeName="kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814291    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bae499a-6abf-4f3a-8e0b-9bb821884f74" volumeName="kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814302    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f9fcff5-40fb-422e-9319-50a1c4156131" volumeName="kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814313    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f" volumeName="kubernetes.io/empty-dir/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-tmp" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814332    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f" volumeName="kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814344    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f3ef0c70-9090-44d0-bb59-95ab1cf0c245" volumeName="kubernetes.io/projected/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kube-api-access-ph67d" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814364    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2" volumeName="kubernetes.io/projected/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-kube-api-access-sdx5q" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814377    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9701f603-d98d-49b5-adc7-3550bdab41d2" volumeName="kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814386    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f" volumeName="kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814396    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6f0136b-88a1-427c-92bc-5cc7ed9d7030" volumeName="kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-daemon-config" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814405    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6f0136b-88a1-427c-92bc-5cc7ed9d7030" volumeName="kubernetes.io/projected/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-kube-api-access-64j65" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814418    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb7545c5-92b9-4107-93d0-f0617277df0f" volumeName="kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-config" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814429    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9701f603-d98d-49b5-adc7-3550bdab41d2" volumeName="kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814440    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21192d3f-1a3f-4a32-ab4a-70fbcada9424" volumeName="kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-tuned" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814451    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="84f3a1bb-9373-443e-b957-1bf6ccb03477" volumeName="kubernetes.io/configmap/84f3a1bb-9373-443e-b957-1bf6ccb03477-serviceca" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814461    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8db49c21-a7b4-4fc6-8342-95243844e22f" volumeName="kubernetes.io/projected/8db49c21-a7b4-4fc6-8342-95243844e22f-kube-api-access-xtppr" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814472    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f" volumeName="kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814482    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1" volumeName="kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814494    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9701f603-d98d-49b5-adc7-3550bdab41d2" volumeName="kubernetes.io/projected/9701f603-d98d-49b5-adc7-3550bdab41d2-kube-api-access-rmk4b" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814504    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9701f603-d98d-49b5-adc7-3550bdab41d2" volumeName="kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-federate-client-tls" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814514    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f" volumeName="kubernetes.io/configmap/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814526    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21192d3f-1a3f-4a32-ab4a-70fbcada9424" volumeName="kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-tmp" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814537    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2" volumeName="kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-tls" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814556    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="376fe364-ccc4-4435-b5e0-bef1e6715b26" volumeName="kubernetes.io/configmap/376fe364-ccc4-4435-b5e0-bef1e6715b26-config-volume" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814568    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bae499a-6abf-4f3a-8e0b-9bb821884f74" volumeName="kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814577    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5f9fcff5-40fb-422e-9319-50a1c4156131" volumeName="kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-binary-copy" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814588    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8db49c21-a7b4-4fc6-8342-95243844e22f" volumeName="kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814598    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e972ba1-2295-49d0-998b-00b86a1737af" volumeName="kubernetes.io/secret/4e972ba1-2295-49d0-998b-00b86a1737af-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814610    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8db49c21-a7b4-4fc6-8342-95243844e22f" volumeName="kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814617    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a410d5a1-3091-405c-a1b2-9d790b77b458" volumeName="kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814625    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f" volumeName="kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814631    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f" volumeName="kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-grpc-tls" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814639    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd11fe2e-49c0-4478-928a-a04ed1f29273" volumeName="kubernetes.io/projected/cd11fe2e-49c0-4478-928a-a04ed1f29273-kube-api-access-wcj9x" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814646    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6f0136b-88a1-427c-92bc-5cc7ed9d7030" volumeName="kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cni-binary-copy" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814653    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="376fe364-ccc4-4435-b5e0-bef1e6715b26" volumeName="kubernetes.io/secret/376fe364-ccc4-4435-b5e0-bef1e6715b26-metrics-tls" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814679    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e972ba1-2295-49d0-998b-00b86a1737af" volumeName="kubernetes.io/empty-dir/4e972ba1-2295-49d0-998b-00b86a1737af-data-volume" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814690    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8db49c21-a7b4-4fc6-8342-95243844e22f" volumeName="kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814697    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f" volumeName="kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814704    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd11fe2e-49c0-4478-928a-a04ed1f29273" volumeName="kubernetes.io/secret/cd11fe2e-49c0-4478-928a-a04ed1f29273-cert" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814712    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d967b5ff-61dc-4c9a-8705-d735e0075db5" volumeName="kubernetes.io/configmap/d967b5ff-61dc-4c9a-8705-d735e0075db5-iptables-alerter-script" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814718    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb7545c5-92b9-4107-93d0-f0617277df0f" volumeName="kubernetes.io/projected/fb7545c5-92b9-4107-93d0-f0617277df0f-kube-api-access-4cb74" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814726    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="376fe364-ccc4-4435-b5e0-bef1e6715b26" volumeName="kubernetes.io/empty-dir/376fe364-ccc4-4435-b5e0-bef1e6715b26-tmp-dir" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814733    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e972ba1-2295-49d0-998b-00b86a1737af" volumeName="kubernetes.io/projected/4e972ba1-2295-49d0-998b-00b86a1737af-kube-api-access-5wwpn" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814741    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8db49c21-a7b4-4fc6-8342-95243844e22f" volumeName="kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814748    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb7545c5-92b9-4107-93d0-f0617277df0f" volumeName="kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814755    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8db49c21-a7b4-4fc6-8342-95243844e22f" volumeName="kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-client-ca-bundle" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814762    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9115c245-4790-4124-a435-24b667fc1744" volumeName="kubernetes.io/secret/9115c245-4790-4124-a435-24b667fc1744-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814769    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9701f603-d98d-49b5-adc7-3550bdab41d2" volumeName="kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814777    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9701f603-d98d-49b5-adc7-3550bdab41d2" volumeName="kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814784    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d967b5ff-61dc-4c9a-8705-d735e0075db5" volumeName="kubernetes.io/projected/d967b5ff-61dc-4c9a-8705-d735e0075db5-kube-api-access-nltc9" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814791    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2" volumeName="kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814798    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3bae499a-6abf-4f3a-8e0b-9bb821884f74" volumeName="kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814804    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cd8a3d9-fbbb-43c0-9920-345d2ab627c3" volumeName="kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814812    5503 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb7545c5-92b9-4107-93d0-f0617277df0f" volumeName="kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-env-overrides" seLinuxMountContext=""
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814819    5503 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 22:19:17.816285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.814824    5503 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 22:19:17.823502 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.823472    5503 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 22:19:17.823590 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.823532    5503 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:19:17.823980 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.823956    5503 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:19:17.827137 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.827118    5503 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:19:17.846922 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.846895    5503 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 22:19:17.848787 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.848553    5503 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 22:19:17.848787 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.848577    5503 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 22:19:17.848787 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.848599    5503 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 22:19:17.851723 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:19:17.849370    5503 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 22:19:17.857375 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.857362    5503 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:19:17.863717 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.863555    5503 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="381e0671943f5d7a3d56ce16fea521894f49d70e6c993cc850b76aaf8c1f7d96" exitCode=0
Oct 30 22:19:17.863717 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.863720    5503 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="78b671f692a8ebc3b97adcdd370a493cb80a749c224c2e181c4c680eaf085d86" exitCode=0
Oct 30 22:19:17.863866 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.863730    5503 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="2fbfd55a6b2e83e764f5683041155809d3d5359d6023ed63cee555af4cd84dca" exitCode=0
Oct 30 22:19:17.863866 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.863738    5503 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="1518d088ed8231415e441f620043a4e425b4e852f9a792b2508e60db1517c6d7" exitCode=0
Oct 30 22:19:17.863866 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.863745    5503 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="ef572db646a056859c8a261de74d1b1378a5fbe30739449a19821d555d2177f9" exitCode=0
Oct 30 22:19:17.863866 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.863752    5503 generic.go:356] "Generic (PLEG): container finished" podID="5f9fcff5-40fb-422e-9319-50a1c4156131" containerID="564efb71befef180eb96f6e05e7c8c1f1e9e1b0d0a8cc5a9b72c60b4ad1e8c76" exitCode=0
Oct 30 22:19:17.881165 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.881141    5503 generic.go:356] "Generic (PLEG): container finished" podID="2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2" containerID="d776ac4f78b5b757604701f0a9a8c850d2ed9d85bf25efa56ebfb61b706b6a45" exitCode=0
Oct 30 22:19:17.893772 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.893743    5503 generic.go:356] "Generic (PLEG): container finished" podID="6eb0646618f7571669a1ed8b2da1b80f" containerID="0cfaa3a5781b78d2d096f38ae07c0a939c4933a312148f4255cf0c32d2755e08" exitCode=0
Oct 30 22:19:17.918891 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.918874    5503 manager.go:324] Recovery completed
Oct 30 22:19:17.940105 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940042    5503 cpu_manager.go:226] "Starting" policy="none"
Oct 30 22:19:17.940296 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940282    5503 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 22:19:17.940331 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940308    5503 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 22:19:17.940434 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940421    5503 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 22:19:17.940464 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940435    5503 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 22:19:17.940464 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940456    5503 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 22:19:17.940464 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940463    5503 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 22:19:17.940541 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940470    5503 policy_none.go:50] "Start"
Oct 30 22:19:17.940541 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940482    5503 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 22:19:17.940541 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940493    5503 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 22:19:17.940626 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940617    5503 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 22:19:17.940655 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940627    5503 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 22:19:17.940655 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.940633    5503 policy_none.go:44] "Start"
Oct 30 22:19:17.945233 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.945210    5503 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 22:19:17.947856 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.947841    5503 manager.go:341] "Starting Device Plugin manager"
Oct 30 22:19:17.948342 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:19:17.947865    5503 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 22:19:17.948390 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.948347    5503 server.go:87] "Starting device plugin registration server"
Oct 30 22:19:17.948592 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.948582    5503 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 22:19:17.948629 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.948593    5503 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 22:19:17.948734 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.948711    5503 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 22:19:17.948845 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.948815    5503 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 22:19:17.948903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.948850    5503 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 22:19:17.948903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.948854    5503 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 22:19:17.949000 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.948897    5503 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:19:17.948841851Z","UUID":"fbdafe0e-817b-4e24-beb3-4eadc2acec9a","Handler":null,"Name":"","Endpoint":""}
Oct 30 22:19:17.950264 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.950211    5503 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"]
Oct 30 22:19:17.950685 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:19:17.950647    5503 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 22:19:17.950856 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.950839    5503 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 22:19:17.950934 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.950867    5503 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 22:19:17.951383 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.951109    5503 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:17.951383 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.951107    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"e09f6b9a488f34d09a2deb5cc4fd598a","Type":"ContainerStarted","Data":"8fcd83ab29f9d728d838c745e96eec4f2489768936a15a1a7133d86f15b96f89"}
Oct 30 22:19:17.951383 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.951150    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"e09f6b9a488f34d09a2deb5cc4fd598a","Type":"ContainerStarted","Data":"6feaa293f879d3744c71c3a5cd4fbabb405776faf2c1b3a13198227247b500ad"}
Oct 30 22:19:17.951383 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.951202    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"6eb0646618f7571669a1ed8b2da1b80f","Type":"ContainerStarted","Data":"39c7b37b69aaa2b310890904d2aa7a55501499f42b4a4e294e88f4328d375b14"}
Oct 30 22:19:17.951383 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.951221    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"6eb0646618f7571669a1ed8b2da1b80f","Type":"ContainerDied","Data":"0cfaa3a5781b78d2d096f38ae07c0a939c4933a312148f4255cf0c32d2755e08"}
Oct 30 22:19:17.951383 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.951239    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal" event={"ID":"6eb0646618f7571669a1ed8b2da1b80f","Type":"ContainerStarted","Data":"1a879aa3adc402f5517ef9cc23434f9de1f238b5ee9e9d36d908604df3c0665e"}
Oct 30 22:19:17.951646 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.951465    5503 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:17.965327 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.965309    5503 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:19:17.965393 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:19:17.965347    5503 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:17.973981 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:17.973962    5503 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:19:17.974043 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:19:17.973993    5503 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.023762 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.023743    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.023840 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.023769    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e09f6b9a488f34d09a2deb5cc4fd598a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"e09f6b9a488f34d09a2deb5cc4fd598a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.023840 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.023785    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.048975 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.048956    5503 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:19:18.051035 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.051016    5503 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:19:18.051112 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.051050    5503 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:19:18.051112 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.051065    5503 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-128-97.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:19:18.051203 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.051115    5503 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.066212 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.066192    5503 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.066299 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.066248    5503 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.124021 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.124001    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.124106 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.124027    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.124106 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.124042    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e09f6b9a488f34d09a2deb5cc4fd598a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"e09f6b9a488f34d09a2deb5cc4fd598a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.124106 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.124058    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.124106 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.124074    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e09f6b9a488f34d09a2deb5cc4fd598a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"e09f6b9a488f34d09a2deb5cc4fd598a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.124248 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.124115    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6eb0646618f7571669a1ed8b2da1b80f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" (UID: \"6eb0646618f7571669a1ed8b2da1b80f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.797975 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.797951    5503 apiserver.go:52] "Watching apiserver"
Oct 30 22:19:18.808957 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.808940    5503 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 22:19:18.809458 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.809440    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-gtwvr","kube-system/konnectivity-agent-cmh6h","openshift-dns/dns-default-96mpc","openshift-multus/network-metrics-daemon-4pvll","openshift-ingress-canary/ingress-canary-hrpz9","openshift-console/console-7ddcd679db-gbd7j","openshift-insights/insights-runtime-extractor-5rttv","openshift-cluster-node-tuning-operator/tuned-7wqr7","openshift-monitoring/monitoring-plugin-77f4f76594-249ck","openshift-monitoring/node-exporter-jxc4l","kube-system/kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal","openshift-network-operator/iptables-alerter-5f2j9","openshift-ovn-kubernetes/ovnkube-node-82mjm","openshift-image-registry/node-ca-dkmz4","openshift-multus/multus-4gvnm","kube-system/global-pull-secret-syncer-hgsqv","openshift-monitoring/thanos-querier-6776c6b778-2gtz5","openshift-dns/node-resolver-wqsgd","openshift-monitoring/metrics-server-5f5784d99-gnbws","openshift-monitoring/telemeter-client-56c84f9555-d5rqg","openshift-multus/multus-additional-cni-plugins-kl9nq","openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"]
Oct 30 22:19:18.815131 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815106    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.815382 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815366    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 22:19:18.815494 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815474    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.815545 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815492    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 22:19:18.815616 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815602    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.815660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815637    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.815752 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815738    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 22:19:18.815796 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815750    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 22:19:18.815844 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815827    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.815888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815860    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.815934 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.815910    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.816083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.816065    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.816227 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.816212    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.816283 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.816254    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 22:19:18.816381 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.816361    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.816691 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.816653    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 22:19:18.816870 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.816739    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 22:19:18.816954 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.816905    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 22:19:18.817120 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.817099    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.817185 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.817127    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 22:19:18.817230 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.817215    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.822623 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.822601    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.822724 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.822696    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.822771 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.822696    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.823111 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.823086    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.823142 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.823110    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 22:19:18.823142 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.823097    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 22:19:18.823142 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.823132    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.826244 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.826232    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.827368 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827348    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-system-cni-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.827452 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827382    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjz\" (UniqueName: \"kubernetes.io/projected/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-kube-api-access-7xdjz\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:19:18.827452 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827431    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmhgd\" (UniqueName: \"kubernetes.io/projected/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-kube-api-access-hmhgd\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:19:18.827525 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827459    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-kubelet-config\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:18.827525 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827487    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npk9p\" (UniqueName: \"kubernetes.io/projected/5f9fcff5-40fb-422e-9319-50a1c4156131-kube-api-access-npk9p\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.827525 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827512    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:19:18.827624 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827535    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-sys\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.827680 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827626    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.827722 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827703    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-tuned\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.827753 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827734    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:18.827787 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827765    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-cnibin\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.827787 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827767    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-tuned\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.827853 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827805    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.827853 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827838    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-dbus\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:18.827907 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827885    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-lib-modules\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.827949 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827932    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-kubernetes\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.827994 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.827981    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-conf\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828030 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828007    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-run\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828030 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828013    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.828090 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828038    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-tmp\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828090 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828076    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-hosts-file\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:19:18.828150 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828128    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-tmp-dir\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:19:18.828150 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828130    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21192d3f-1a3f-4a32-ab4a-70fbcada9424-tmp\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828153    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-modprobe-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828168    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-host\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828182    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-tmp-dir\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:19:18.828207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828183    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-668ht\" (UniqueName: \"kubernetes.io/projected/21192d3f-1a3f-4a32-ab4a-70fbcada9424-kube-api-access-668ht\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828365 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828221    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-binary-copy\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.828365 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828308    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.828429 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828400    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysconfig\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828429 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828418    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-systemd\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828492 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828451    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-binary-copy\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.828492 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828466    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.828553 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828488    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:19:18.828553 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828518    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-var-lib-kubelet\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.828645 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.828628    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5f9fcff5-40fb-422e-9319-50a1c4156131-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.840193 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.840174    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdjz\" (UniqueName: \"kubernetes.io/projected/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-kube-api-access-7xdjz\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:19:18.840304 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.840202    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmhgd\" (UniqueName: \"kubernetes.io/projected/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-kube-api-access-hmhgd\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:19:18.840779 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.840758    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npk9p\" (UniqueName: \"kubernetes.io/projected/5f9fcff5-40fb-422e-9319-50a1c4156131-kube-api-access-npk9p\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.840819 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.840780    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgg87\" (UniqueName: \"kubernetes.io/projected/7cd8a3d9-fbbb-43c0-9920-345d2ab627c3-kube-api-access-xgg87\") pod \"network-check-target-gtwvr\" (UID: \"7cd8a3d9-fbbb-43c0-9920-345d2ab627c3\") " pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:19:18.841140 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.841120    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-668ht\" (UniqueName: \"kubernetes.io/projected/21192d3f-1a3f-4a32-ab4a-70fbcada9424-kube-api-access-668ht\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.880412 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.880390    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 22:19:18.880412 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.880392    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 22:19:18.880704 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.880686    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 22:19:18.881105 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881084    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-glgqx\""
Oct 30 22:19:18.881221 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881135    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tp94j\""
Oct 30 22:19:18.881221 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881142    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 22:19:18.881221 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881170    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dncrj\""
Oct 30 22:19:18.881398 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881297    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 22:19:18.881398 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881350    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hx8r9\""
Oct 30 22:19:18.881398 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881365    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 22:19:18.881398 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881389    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fal81smpff8pm\""
Oct 30 22:19:18.881559 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881403    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9jqsq\""
Oct 30 22:19:18.881559 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881551    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 22:19:18.881645 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881559    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 22:19:18.881645 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881570    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 22:19:18.881645 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881580    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 22:19:18.881645 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881599    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 22:19:18.881645 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881612    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 22:19:18.881911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881754    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 22:19:18.881911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881790    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-sdfwl\""
Oct 30 22:19:18.881911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881808    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 22:19:18.881911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881819    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jmj7l\""
Oct 30 22:19:18.881911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881837    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qwcf4\""
Oct 30 22:19:18.881911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881912    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kkwnr\""
Oct 30 22:19:18.882256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.881932    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms5nl\""
Oct 30 22:19:18.882256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882032    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 22:19:18.882256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882050    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-szxgd\""
Oct 30 22:19:18.882256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882095    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 22:19:18.882256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882234    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w24gg\""
Oct 30 22:19:18.882256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882246    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882254    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882331    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-jfcbv\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882344    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882359    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882381    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-l75wj\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882436    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882445    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882471    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-srblb\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882477    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-r4dw4\""
Oct 30 22:19:18.882505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882495    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-lrk6p\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882609    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gb6fw\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882646    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882688    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882698    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bh4rp\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882701    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jb8sb\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882720    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2j6f8qkq0ohq5\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882715    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882723    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-4x7f7\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882759    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jm6gb\""
Oct 30 22:19:18.882850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.882762    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 22:19:18.888277 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.888257    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1ae3621a-b431-4675-8617-37461922d241-original-pull-secret\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:18.888507 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.888473    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1-metrics-certs\") pod \"network-metrics-daemon-4pvll\" (UID: \"e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1\") " pod="openshift-multus/network-metrics-daemon-4pvll"
Oct 30 22:19:18.911901 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.911883    5503 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.912472 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.912457    5503 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 22:19:18.914682 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.914638    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.914844 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.914827    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.914921 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.914859    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.914921 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.914902    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 22:19:18.914987 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.914947    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 22:19:18.915050 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915036    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 22:19:18.915114 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915058    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.915114 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915087    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 22:19:18.915114 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915090    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 22:19:18.915252 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915142    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 22:19:18.915252 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915140    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 22:19:18.915252 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915170    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 22:19:18.915400 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915301    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 22:19:18.915400 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915332    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 22:19:18.915537 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.915521    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 22:19:18.917767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.917748    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 22:19:18.917817 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.917766    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 22:19:18.929244 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929225    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.929371 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929253    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-sys-fs\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:18.929371 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929270    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48zdx\" (UniqueName: \"kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.929371 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929284    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-etc-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.929371 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929320    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-env-overrides\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.929371 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929351    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.929603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929486    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.929603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929522    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-systemd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.929603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929548    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcj9x\" (UniqueName: \"kubernetes.io/projected/cd11fe2e-49c0-4478-928a-a04ed1f29273-kube-api-access-wcj9x\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:19:18.929603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929571    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-var-lib-kubelet\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.929603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929581    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-env-overrides\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.929603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929572    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.929603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929566    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-trusted-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929617    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq6r9\" (UniqueName: \"kubernetes.io/projected/9115c245-4790-4124-a435-24b667fc1744-kube-api-access-zq6r9\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929645    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929650    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-var-lib-kubelet\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929651    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929772    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb7545c5-92b9-4107-93d0-f0617277df0f-ovn-node-metrics-cert\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929798    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-system-cni-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929809    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929825    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-bin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929849    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-conf-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929873    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-var-lib-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.929908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929892    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-system-cni-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929915    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd11fe2e-49c0-4478-928a-a04ed1f29273-cert\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.929962    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930008    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930013    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fb7545c5-92b9-4107-93d0-f0617277df0f-ovn-node-metrics-cert\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930037    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64j65\" (UniqueName: \"kubernetes.io/projected/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-kube-api-access-64j65\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930067    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-etc-selinux\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930079    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cd11fe2e-49c0-4478-928a-a04ed1f29273-cert\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930090    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e972ba1-2295-49d0-998b-00b86a1737af-crio-socket\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930131    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-agent-certs\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930152    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.930307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930244    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-conf\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930358    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-sys\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930381    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930406    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-modprobe-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930445    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-agent-certs\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930453    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-conf\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930486    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-ovn\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930548    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-modprobe-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930575    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-root\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930593    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-netns\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930630    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-hostroot\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.930660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930649    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-script-lib\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930698    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cni-binary-copy\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930720    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-log-socket\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930867    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cni-binary-copy\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930879    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930911    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-tmp\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930944    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-textfile\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930964    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.930978    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-script-lib\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931022    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-textfile\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931054    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8db49c21-a7b4-4fc6-8342-95243844e22f-audit-log\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.931133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931096    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-tmp\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.931833 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931228    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.931833 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931239    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-socket-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:18.931833 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931271    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8db49c21-a7b4-4fc6-8342-95243844e22f-audit-log\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.931833 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931308    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:19:18.931833 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931715    5503 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:19:18.931833 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931718    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-slash\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.931833 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931756    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.931833 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931789    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89prq\" (UniqueName: \"kubernetes.io/projected/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-kube-api-access-89prq\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.931833 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:19:18.931789    5503 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-97.us-east-2.compute.internal"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931942    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.931996    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-serving-certs-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932093    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-multus\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932115    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-kubelet\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932095    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932180    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-metrics-client-ca\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932211    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-konnectivity-ca\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932241    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-system-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932239    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-serving-certs-ca-bundle\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.932261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932261    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-cnibin\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932284    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-dbus\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932321    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/376fe364-ccc4-4435-b5e0-bef1e6715b26-metrics-tls\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932366    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-metrics-client-ca\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932399    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nltc9\" (UniqueName: \"kubernetes.io/projected/d967b5ff-61dc-4c9a-8705-d735e0075db5-kube-api-access-nltc9\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932406    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-dbus\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932428    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f0f05f12-85f1-4c1d-9115-e1cd2a336b55-konnectivity-ca\") pod \"konnectivity-agent-cmh6h\" (UID: \"f0f05f12-85f1-4c1d-9115-e1cd2a336b55\") " pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932432    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-cnibin\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932452    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932469    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932513    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932519    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/376fe364-ccc4-4435-b5e0-bef1e6715b26-metrics-tls\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932540    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-lib-modules\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932562    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-kubernetes\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932584    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-run\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932606    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-hosts-file\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932629    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86xfb\" (UniqueName: \"kubernetes.io/projected/376fe364-ccc4-4435-b5e0-bef1e6715b26-kube-api-access-86xfb\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932650    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932654    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-bin\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932687    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-lib-modules\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932705    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932709    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e972ba1-2295-49d0-998b-00b86a1737af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932703    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.932749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932741    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-host\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932767    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-tls\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932771    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b6fa19a-6fb7-40b6-9073-0a52cfca6b79-hosts-file\") pod \"node-resolver-wqsgd\" (UID: \"0b6fa19a-6fb7-40b6-9073-0a52cfca6b79\") " pod="openshift-dns/node-resolver-wqsgd"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932794    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cnibin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932816    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-run\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932819    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph67d\" (UniqueName: \"kubernetes.io/projected/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kube-api-access-ph67d\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932853    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysconfig\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932861    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e972ba1-2295-49d0-998b-00b86a1737af-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932877    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932903    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-registration-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932741    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-kubernetes\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932928    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysconfig\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.932934    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cb74\" (UniqueName: \"kubernetes.io/projected/fb7545c5-92b9-4107-93d0-f0617277df0f-kube-api-access-4cb74\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933047    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wwpn\" (UniqueName: \"kubernetes.io/projected/4e972ba1-2295-49d0-998b-00b86a1737af-kube-api-access-5wwpn\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933079    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-client-certs\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933096    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-tls\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933096    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5f9fcff5-40fb-422e-9319-50a1c4156131-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kl9nq\" (UID: \"5f9fcff5-40fb-422e-9319-50a1c4156131\") " pod="openshift-multus/multus-additional-cni-plugins-kl9nq"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933122    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-kubelet\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933133    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-host\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933149    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-etc-kubernetes\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933178    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84f3a1bb-9373-443e-b957-1bf6ccb03477-host\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933231    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933260    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-systemd-units\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933282    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-kubelet-config\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933329    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/376fe364-ccc4-4435-b5e0-bef1e6715b26-config-volume\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933340    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-client-certs\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933345    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/376fe364-ccc4-4435-b5e0-bef1e6715b26-tmp-dir\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933389    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933401    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1ae3621a-b431-4675-8617-37461922d241-kubelet-config\") pod \"global-pull-secret-syncer-hgsqv\" (UID: \"1ae3621a-b431-4675-8617-37461922d241\") " pod="kube-system/global-pull-secret-syncer-hgsqv"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933415    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-client-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933418    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/376fe364-ccc4-4435-b5e0-bef1e6715b26-tmp-dir\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933403    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933452    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933481    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933526    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/376fe364-ccc4-4435-b5e0-bef1e6715b26-config-volume\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933525    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-wtmp\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933542    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-accelerators-collector-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933630    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-client-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933634    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933685    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmk4b\" (UniqueName: \"kubernetes.io/projected/9701f603-d98d-49b5-adc7-3550bdab41d2-kube-api-access-rmk4b\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933716    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-tls\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933728    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/a410d5a1-3091-405c-a1b2-9d790b77b458-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:19:18.933750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933739    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a410d5a1-3091-405c-a1b2-9d790b77b458-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933784    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933810    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933835    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-config\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933842    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-secret-telemeter-client\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.933921    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8db49c21-a7b4-4fc6-8342-95243844e22f-secret-metrics-server-tls\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934014    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934015    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-sys\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934048    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934061    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-sys\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934020    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fb7545c5-92b9-4107-93d0-f0617277df0f-ovnkube-config\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934073    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d967b5ff-61dc-4c9a-8705-d735e0075db5-host-slash\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934124    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934153    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a410d5a1-3091-405c-a1b2-9d790b77b458-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934182    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-netns\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934222    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-node-log\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934243    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-sysctl-d\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934263    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-netd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934300    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e972ba1-2295-49d0-998b-00b86a1737af-data-volume\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934305    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934351    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84f3a1bb-9373-443e-b957-1bf6ccb03477-serviceca\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934369    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e972ba1-2295-49d0-998b-00b86a1737af-data-volume\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934421    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdx5q\" (UniqueName: \"kubernetes.io/projected/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-kube-api-access-sdx5q\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934456    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-daemon-config\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934480    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-systemd\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934653    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-daemon-config\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934718    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e972ba1-2295-49d0-998b-00b86a1737af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934743    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-grpc-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934774    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-federate-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.934977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.934983    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/84f3a1bb-9373-443e-b957-1bf6ccb03477-serviceca\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935011    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-secret-grpc-tls\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935036    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/21192d3f-1a3f-4a32-ab4a-70fbcada9424-etc-systemd\") pod \"tuned-7wqr7\" (UID: \"21192d3f-1a3f-4a32-ab4a-70fbcada9424\") " pod="openshift-cluster-node-tuning-operator/tuned-7wqr7"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935065    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-metrics-client-ca\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935090    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-k8s-cni-cncf-io\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935115    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-multus-certs\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935154    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935182    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d967b5ff-61dc-4c9a-8705-d735e0075db5-iptables-alerter-script\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935362    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d967b5ff-61dc-4c9a-8705-d735e0075db5-iptables-alerter-script\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935379    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e972ba1-2295-49d0-998b-00b86a1737af-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935407    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtppr\" (UniqueName: \"kubernetes.io/projected/8db49c21-a7b4-4fc6-8342-95243844e22f-kube-api-access-xtppr\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935432    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwgtv\" (UniqueName: \"kubernetes.io/projected/a410d5a1-3091-405c-a1b2-9d790b77b458-kube-api-access-pwgtv\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935477    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935529    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935555    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-federate-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935556    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-socket-dir-parent\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935582    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935731    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9701f603-d98d-49b5-adc7-3550bdab41d2-metrics-client-ca\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935792    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/9701f603-d98d-49b5-adc7-3550bdab41d2-telemeter-client-tls\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935904    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.935993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.935910    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.936906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.936091    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.936906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.936134    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-metrics-client-ca\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.936906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.936208    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-device-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:18.936906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.936247    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9115c245-4790-4124-a435-24b667fc1744-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:18.936906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.936273    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-metrics-server-audit-profiles\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.936906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.936312    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm9gc\" (UniqueName: \"kubernetes.io/projected/84f3a1bb-9373-443e-b957-1bf6ccb03477-kube-api-access-gm9gc\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:19:18.936906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.936405    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-metrics-client-ca\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.936906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.936453    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9115c245-4790-4124-a435-24b667fc1744-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:18.936906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.936746    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8db49c21-a7b4-4fc6-8342-95243844e22f-metrics-server-audit-profiles\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:18.941336 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.941313    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq6r9\" (UniqueName: \"kubernetes.io/projected/9115c245-4790-4124-a435-24b667fc1744-kube-api-access-zq6r9\") pod \"monitoring-plugin-77f4f76594-249ck\" (UID: \"9115c245-4790-4124-a435-24b667fc1744\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:18.941426 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.941321    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64j65\" (UniqueName: \"kubernetes.io/projected/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-kube-api-access-64j65\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:18.941426 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.941358    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48zdx\" (UniqueName: \"kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx\") pod \"console-7ddcd679db-gbd7j\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") " pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:18.941487 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.941451    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcj9x\" (UniqueName: \"kubernetes.io/projected/cd11fe2e-49c0-4478-928a-a04ed1f29273-kube-api-access-wcj9x\") pod \"ingress-canary-hrpz9\" (UID: \"cd11fe2e-49c0-4478-928a-a04ed1f29273\") " pod="openshift-ingress-canary/ingress-canary-hrpz9"
Oct 30 22:19:18.943356 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.943336    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-89prq\" (UniqueName: \"kubernetes.io/projected/c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f-kube-api-access-89prq\") pod \"thanos-querier-6776c6b778-2gtz5\" (UID: \"c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f\") " pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:18.944513 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.944232    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86xfb\" (UniqueName: \"kubernetes.io/projected/376fe364-ccc4-4435-b5e0-bef1e6715b26-kube-api-access-86xfb\") pod \"dns-default-96mpc\" (UID: \"376fe364-ccc4-4435-b5e0-bef1e6715b26\") " pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:18.944513 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.944253    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wwpn\" (UniqueName: \"kubernetes.io/projected/4e972ba1-2295-49d0-998b-00b86a1737af-kube-api-access-5wwpn\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:18.944513 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.944478    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nltc9\" (UniqueName: \"kubernetes.io/projected/d967b5ff-61dc-4c9a-8705-d735e0075db5-kube-api-access-nltc9\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:19:18.944630 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.944545    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph67d\" (UniqueName: \"kubernetes.io/projected/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kube-api-access-ph67d\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:18.944765 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.944751    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmk4b\" (UniqueName: \"kubernetes.io/projected/9701f603-d98d-49b5-adc7-3550bdab41d2-kube-api-access-rmk4b\") pod \"telemeter-client-56c84f9555-d5rqg\" (UID: \"9701f603-d98d-49b5-adc7-3550bdab41d2\") " pod="openshift-monitoring/telemeter-client-56c84f9555-d5rqg"
Oct 30 22:19:18.946095 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.946079    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cb74\" (UniqueName: \"kubernetes.io/projected/fb7545c5-92b9-4107-93d0-f0617277df0f-kube-api-access-4cb74\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:18.947917 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.947888    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdx5q\" (UniqueName: \"kubernetes.io/projected/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-kube-api-access-sdx5q\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:18.948479 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.948457    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm9gc\" (UniqueName: \"kubernetes.io/projected/84f3a1bb-9373-443e-b957-1bf6ccb03477-kube-api-access-gm9gc\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:19:18.948586 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.948568    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwgtv\" (UniqueName: \"kubernetes.io/projected/a410d5a1-3091-405c-a1b2-9d790b77b458-kube-api-access-pwgtv\") pod \"prometheus-operator-9cd89b74c-9dp62\" (UID: \"a410d5a1-3091-405c-a1b2-9d790b77b458\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-9dp62"
Oct 30 22:19:18.948847 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:18.948827    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtppr\" (UniqueName: \"kubernetes.io/projected/8db49c21-a7b4-4fc6-8342-95243844e22f-kube-api-access-xtppr\") pod \"metrics-server-5f5784d99-gnbws\" (UID: \"8db49c21-a7b4-4fc6-8342-95243844e22f\") " pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:19.036609 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036580    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-log-socket\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.036756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036614    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.036756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036635    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-socket-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.036756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036658    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-slash\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.036756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036696    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-log-socket\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.036756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036698    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-multus\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.036756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036722    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.036756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036740    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-kubelet\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.036756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036729    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-multus\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.036756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036757    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-slash\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036730    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-socket-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036772    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-system-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036798    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-bin\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036802    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-kubelet\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036816    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cnibin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036838    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-system-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036855    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-bin\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036859    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-cnibin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036882    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-registration-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036917    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-kubelet\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036940    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-etc-kubernetes\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036949    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-registration-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036961    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84f3a1bb-9373-443e-b957-1bf6ccb03477-host\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.036985    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-systemd-units\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037011    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037021    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-etc-kubernetes\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037031    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/84f3a1bb-9373-443e-b957-1bf6ccb03477-host\") pod \"node-ca-dkmz4\" (UID: \"84f3a1bb-9373-443e-b957-1bf6ccb03477\") " pod="openshift-image-registry/node-ca-dkmz4"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037033    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-kubelet\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037048    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-systemd-units\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037069    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-wtmp\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037072    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-kubelet-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037087    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037110    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d967b5ff-61dc-4c9a-8705-d735e0075db5-host-slash\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037122    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-node-exporter-wtmp\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037127    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-netns\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.037148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037148    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-node-log\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037170    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-netd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037199    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-run-netns\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037210    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d967b5ff-61dc-4c9a-8705-d735e0075db5-host-slash\") pod \"iptables-alerter-5f2j9\" (UID: \"d967b5ff-61dc-4c9a-8705-d735e0075db5\") " pod="openshift-network-operator/iptables-alerter-5f2j9"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037216    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-cni-netd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037236    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037242    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-node-log\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037288    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-k8s-cni-cncf-io\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037318    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-multus-certs\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037345    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-socket-dir-parent\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037342    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-k8s-cni-cncf-io\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037371    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-device-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037377    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-multus-certs\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037396    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-sys-fs\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037415    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-device-dir\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037418    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-etc-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037434    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-socket-dir-parent\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037443    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-systemd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037464    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-etc-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037450    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-sys-fs\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037481    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-systemd\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037483    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-bin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037511    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-var-lib-cni-bin\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037512    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-conf-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037572    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-var-lib-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037550    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-conf-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037597    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-var-lib-openvswitch\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037600    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037620    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-etc-selinux\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037636    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e972ba1-2295-49d0-998b-00b86a1737af-crio-socket\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037647    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3ef0c70-9090-44d0-bb59-95ab1cf0c245-etc-selinux\") pod \"aws-ebs-csi-driver-node-h8xmf\" (UID: \"f3ef0c70-9090-44d0-bb59-95ab1cf0c245\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-h8xmf"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037654    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-sys\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037687    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037697    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e972ba1-2295-49d0-998b-00b86a1737af-crio-socket\") pod \"insights-runtime-extractor-5rttv\" (UID: \"4e972ba1-2295-49d0-998b-00b86a1737af\") " pod="openshift-insights/insights-runtime-extractor-5rttv"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037702    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-ovn\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037709    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-sys\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037720    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-root\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037723    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-run-ovn\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037620    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb7545c5-92b9-4107-93d0-f0617277df0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-82mjm\" (UID: \"fb7545c5-92b9-4107-93d0-f0617277df0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037734    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-netns\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037752    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2-root\") pod \"node-exporter-jxc4l\" (UID: \"2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2\") " pod="openshift-monitoring/node-exporter-jxc4l"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037769    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-multus-cni-dir\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037756    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-hostroot\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037774    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-host-run-netns\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.038041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.037778    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f6f0136b-88a1-427c-92bc-5cc7ed9d7030-hostroot\") pod \"multus-4gvnm\" (UID: \"f6f0136b-88a1-427c-92bc-5cc7ed9d7030\") " pod="openshift-multus/multus-4gvnm"
Oct 30 22:19:19.460210 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.460184    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:19:19.801873 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.801768    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:19.914745 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.914719    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:19.918246 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.918223    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-249ck"
Oct 30 22:19:19.947499 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:19.947473    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:20.237161 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:20.237136    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-96mpc"
Oct 30 22:19:21.128056 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:21.128033    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6776c6b778-2gtz5"
Oct 30 22:19:23.041681 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:23.041640    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:23.141489 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:23.141460    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:23.234411 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:23.234390    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:23.373708 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:23.373680    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:23.459820 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:23.459799    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:24.707695 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:24.707652    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:27.033435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:27.033366    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:27.298937 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:27.298874    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:27.451290 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:27.451267    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:28.239113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:28.239087    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:28.604599 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:28.604572    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:28.604768 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:28.604756    5503 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:19:28.785997 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:28.785972    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:29.001255 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:29.001204    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:30.701652 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:30.701616    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:30.797151 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:30.797126    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:32.456858 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:32.456834    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cmh6h"
Oct 30 22:19:33.378938 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:33.378914    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:33.382564 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:33.382532    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:19:33.615986 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:33.615959    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:33.712464 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:33.712410    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:37.961678 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:37.961608    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.059960 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.059928    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.272891 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.272832    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.380281 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.380259    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.427965 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.427939    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b88dcb6c8-v2bvm"]
Oct 30 22:19:38.430420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.430401    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.442252 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.442231    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b88dcb6c8-v2bvm"]
Oct 30 22:19:38.505549 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.505520    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.566724 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.566679    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4356691f-f269-47a4-84d8-c828d47d0b64-console-serving-cert\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.566724 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.566709    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4356691f-f269-47a4-84d8-c828d47d0b64-console-oauth-config\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.566903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.566725    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-service-ca\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.566903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.566748    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-trusted-ca-bundle\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.566903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.566879    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-console-config\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.566903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.566897    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-oauth-serving-cert\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.567087 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.566960    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pmhs\" (UniqueName: \"kubernetes.io/projected/4356691f-f269-47a4-84d8-c828d47d0b64-kube-api-access-4pmhs\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.567087 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.567001    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4356691f-f269-47a4-84d8-c828d47d0b64-tmp\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.601072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.601050    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:38.667511 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.667481    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-trusted-ca-bundle\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.667606 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.667521    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-console-config\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.667606 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.667537    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-oauth-serving-cert\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.667606 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.667553    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pmhs\" (UniqueName: \"kubernetes.io/projected/4356691f-f269-47a4-84d8-c828d47d0b64-kube-api-access-4pmhs\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.667606 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.667575    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4356691f-f269-47a4-84d8-c828d47d0b64-tmp\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.667808 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.667615    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4356691f-f269-47a4-84d8-c828d47d0b64-console-serving-cert\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.667808 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.667648    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4356691f-f269-47a4-84d8-c828d47d0b64-console-oauth-config\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.667808 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.667691    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-service-ca\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.668100 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.668072    5503 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 22:19:38.668100 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.668086    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4356691f-f269-47a4-84d8-c828d47d0b64-tmp\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.668253 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.668240    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-console-config\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.668368 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.668347    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-oauth-serving-cert\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.668368 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.668362    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-trusted-ca-bundle\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.668470 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.668410    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4356691f-f269-47a4-84d8-c828d47d0b64-service-ca\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.670035 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.670019    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4356691f-f269-47a4-84d8-c828d47d0b64-console-oauth-config\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.670211 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.670193    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4356691f-f269-47a4-84d8-c828d47d0b64-console-serving-cert\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.680552 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.680530    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pmhs\" (UniqueName: \"kubernetes.io/projected/4356691f-f269-47a4-84d8-c828d47d0b64-kube-api-access-4pmhs\") pod \"console-6b88dcb6c8-v2bvm\" (UID: \"4356691f-f269-47a4-84d8-c828d47d0b64\") " pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:38.739484 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:38.739463    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:39.011572 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:39.011546    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b88dcb6c8-v2bvm"]
Oct 30 22:19:39.014573 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:39.014543    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4356691f_f269_47a4_84d8_c828d47d0b64.slice/crio-308fd1f21d97f5a03cde818b19750b50cd441065bb5cc69255c0b3eb4ab0ea2a WatchSource:0}: Error finding container 308fd1f21d97f5a03cde818b19750b50cd441065bb5cc69255c0b3eb4ab0ea2a: Status 404 returned error can't find the container with id 308fd1f21d97f5a03cde818b19750b50cd441065bb5cc69255c0b3eb4ab0ea2a
Oct 30 22:19:39.082694 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:39.082655    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:39.181109 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:39.181083    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:39.979762 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:39.979729    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b88dcb6c8-v2bvm" event={"ID":"4356691f-f269-47a4-84d8-c828d47d0b64","Type":"ContainerStarted","Data":"b58ad1db338f29e6c4378818db94a3af2194208445f3cf3809ca2d4b56891597"}
Oct 30 22:19:39.979909 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:39.979800    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b88dcb6c8-v2bvm" event={"ID":"4356691f-f269-47a4-84d8-c828d47d0b64","Type":"ContainerStarted","Data":"308fd1f21d97f5a03cde818b19750b50cd441065bb5cc69255c0b3eb4ab0ea2a"}
Oct 30 22:19:40.002658 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:40.002608    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-6b88dcb6c8-v2bvm" podStartSLOduration=2.002597066 podStartE2EDuration="2.002597066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:40.002002904 +0000 UTC m=+22.271084428" watchObservedRunningTime="2026-10-30 22:19:40.002597066 +0000 UTC m=+22.271678588"
Oct 30 22:19:40.389198 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:40.389159    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:40.488161 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:40.488075    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:44.028586 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.028551    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:44.128854 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.128819    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:19:44.355691 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.355646    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:19:44.360899 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.360880    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.365783 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.365764    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 22:19:44.365900 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.365782    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 22:19:44.365900 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.365797    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 22:19:44.365900 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.365791    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 22:19:44.365900 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.365852    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x6d62\""
Oct 30 22:19:44.366232 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.366209    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 22:19:44.366462 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.366448    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 22:19:44.366547 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.366470    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 22:19:44.367321 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.367303    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 22:19:44.370183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.370165    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 22:19:44.379590 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.379565    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:19:44.507452 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507421    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19fca836-f67e-48a6-a9b6-3060f0261e51-config-out\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507593 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507461    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19fca836-f67e-48a6-a9b6-3060f0261e51-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507593 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507487    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507745 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507589    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19fca836-f67e-48a6-a9b6-3060f0261e51-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507745 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507619    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwkzs\" (UniqueName: \"kubernetes.io/projected/19fca836-f67e-48a6-a9b6-3060f0261e51-kube-api-access-mwkzs\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507745 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507648    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-web-config\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507839 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507755    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-config-volume\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507839 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507785    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507910 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507851    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507910 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507880    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507965 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507911    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19fca836-f67e-48a6-a9b6-3060f0261e51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.507965 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507936    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19fca836-f67e-48a6-a9b6-3060f0261e51-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.508025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.507974    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.608744 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608682    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19fca836-f67e-48a6-a9b6-3060f0261e51-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.608744 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608736    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwkzs\" (UniqueName: \"kubernetes.io/projected/19fca836-f67e-48a6-a9b6-3060f0261e51-kube-api-access-mwkzs\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.608906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608761    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-web-config\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.608906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608787    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-config-volume\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.608906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608813    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.608906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608862    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.608906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608890    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.609144 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608920    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19fca836-f67e-48a6-a9b6-3060f0261e51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.609144 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608944    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19fca836-f67e-48a6-a9b6-3060f0261e51-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.609144 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.608973    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.609144 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.609000    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19fca836-f67e-48a6-a9b6-3060f0261e51-config-out\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.609144 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.609026    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19fca836-f67e-48a6-a9b6-3060f0261e51-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.609144 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.609054    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.609536 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.609473    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19fca836-f67e-48a6-a9b6-3060f0261e51-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.610358 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.610330    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19fca836-f67e-48a6-a9b6-3060f0261e51-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.612073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.611974    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-config-volume\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.612160 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.612068    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.612160 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.612100    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.612285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.612250    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.612344 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.612319    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.612507 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.612476    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/19fca836-f67e-48a6-a9b6-3060f0261e51-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.612786 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.612768    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/19fca836-f67e-48a6-a9b6-3060f0261e51-tls-assets\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.612869 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.612798    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-web-config\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.613316 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.613294    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/19fca836-f67e-48a6-a9b6-3060f0261e51-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.614258 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.614237    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/19fca836-f67e-48a6-a9b6-3060f0261e51-config-out\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.621542 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.621522    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwkzs\" (UniqueName: \"kubernetes.io/projected/19fca836-f67e-48a6-a9b6-3060f0261e51-kube-api-access-mwkzs\") pod \"alertmanager-main-0\" (UID: \"19fca836-f67e-48a6-a9b6-3060f0261e51\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.678958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.678938    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:44.960068 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.960000    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:19:44.964385 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:19:44.964362    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19fca836_f67e_48a6_a9b6_3060f0261e51.slice/crio-4835e508e087a4bfadb011a3f16835abcd23ccfddb0983c90aa3a2591d50b899 WatchSource:0}: Error finding container 4835e508e087a4bfadb011a3f16835abcd23ccfddb0983c90aa3a2591d50b899: Status 404 returned error can't find the container with id 4835e508e087a4bfadb011a3f16835abcd23ccfddb0983c90aa3a2591d50b899
Oct 30 22:19:44.998602 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:44.998571    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19fca836-f67e-48a6-a9b6-3060f0261e51","Type":"ContainerStarted","Data":"4835e508e087a4bfadb011a3f16835abcd23ccfddb0983c90aa3a2591d50b899"}
Oct 30 22:19:45.387077 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:19:45.387046    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:19:46.003268 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:46.003234    5503 generic.go:356] "Generic (PLEG): container finished" podID="19fca836-f67e-48a6-a9b6-3060f0261e51" containerID="137e927d6472a9130fbfee4280041764ed4a9f116e7732ac481b93272663a226" exitCode=0
Oct 30 22:19:46.003430 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:46.003329    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19fca836-f67e-48a6-a9b6-3060f0261e51","Type":"ContainerDied","Data":"137e927d6472a9130fbfee4280041764ed4a9f116e7732ac481b93272663a226"}
Oct 30 22:19:46.004575 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:46.004556    5503 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:19:47.039191 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:47.039164    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:47.043405 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:47.043381    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5f5784d99-gnbws"
Oct 30 22:19:47.610375 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:47.610348    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-82mjm"
Oct 30 22:19:48.013532 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:48.013504    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19fca836-f67e-48a6-a9b6-3060f0261e51","Type":"ContainerStarted","Data":"e422c7107ad30e304b6aceb8fd06d93a39343a351347d6307eb4b8cd2b7bd108"}
Oct 30 22:19:48.013640 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:48.013565    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19fca836-f67e-48a6-a9b6-3060f0261e51","Type":"ContainerStarted","Data":"9f50cde00f84b4ded831074ceccbb121ae3d28de5d4df72251a8433b9c03bf88"}
Oct 30 22:19:48.013640 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:48.013591    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19fca836-f67e-48a6-a9b6-3060f0261e51","Type":"ContainerStarted","Data":"5d735a64e57ad2292b2b3ba30c63516fafdb77a4930f134652da85cc4ee5801b"}
Oct 30 22:19:48.013640 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:48.013615    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19fca836-f67e-48a6-a9b6-3060f0261e51","Type":"ContainerStarted","Data":"9a5f9b770a9603096227deb37451c7a64fabf474603df3a250eed2081e305f7d"}
Oct 30 22:19:48.739833 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:48.739798    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:48.740197 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:48.739849    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:48.744525 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:48.744501    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:49.020394 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:49.020335    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19fca836-f67e-48a6-a9b6-3060f0261e51","Type":"ContainerStarted","Data":"931a8f6c7a21a0910a5c5e6d2c803c3d0dbaae3b80953f048cac2073cc8f1b45"}
Oct 30 22:19:49.020496 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:49.020407    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"19fca836-f67e-48a6-a9b6-3060f0261e51","Type":"ContainerStarted","Data":"70f229c217801aec7aec13eb6de17a094b67abdb9346ea3ad0077a3299585607"}
Oct 30 22:19:49.023744 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:49.023726    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b88dcb6c8-v2bvm"
Oct 30 22:19:49.056186 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:49.056142    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.430378429 podStartE2EDuration="5.05612999s" totalImagesPullingTime="1.611224966s" totalInitContainerRuntime="14.526595ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:49.054207807 +0000 UTC m=+31.323289328" watchObservedRunningTime="2026-10-30 22:19:49.05612999 +0000 UTC m=+31.325211538"
Oct 30 22:19:49.109500 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:49.109470    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ddcd679db-gbd7j"]
Oct 30 22:19:49.917164 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:19:49.917137    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gtwvr"
Oct 30 22:20:00.833272 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:00.833236    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:00.970023 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:00.969996    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:14.129535 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.129502    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-7ddcd679db-gbd7j" podUID="3bae499a-6abf-4f3a-8e0b-9bb821884f74" containerName="console" containerID="cri-o://2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a" gracePeriod=15
Oct 30 22:20:14.361531 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.361510    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:20:14.419995 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.419950    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx\" (UniqueName: \"kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx\") pod \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") "
Oct 30 22:20:14.419995 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.419980    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca\") pod \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") "
Oct 30 22:20:14.420194 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420020    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert\") pod \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") "
Oct 30 22:20:14.420194 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420040    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp\") pod \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") "
Oct 30 22:20:14.420194 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420140    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle\") pod \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") "
Oct 30 22:20:14.420347 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420210    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config\") pod \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") "
Oct 30 22:20:14.420347 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420255    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert\") pod \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") "
Oct 30 22:20:14.420347 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420278    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp" pod "3bae499a-6abf-4f3a-8e0b-9bb821884f74" (UID: "3bae499a-6abf-4f3a-8e0b-9bb821884f74"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:20:14.420347 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420292    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config\") pod \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\" (UID: \"3bae499a-6abf-4f3a-8e0b-9bb821884f74\") "
Oct 30 22:20:14.420539 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420431    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca" pod "3bae499a-6abf-4f3a-8e0b-9bb821884f74" (UID: "3bae499a-6abf-4f3a-8e0b-9bb821884f74"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:14.420597 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420563    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle" pod "3bae499a-6abf-4f3a-8e0b-9bb821884f74" (UID: "3bae499a-6abf-4f3a-8e0b-9bb821884f74"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:14.420721 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420629    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert" pod "3bae499a-6abf-4f3a-8e0b-9bb821884f74" (UID: "3bae499a-6abf-4f3a-8e0b-9bb821884f74"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:14.420721 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420705    5503 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-service-ca\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:14.420721 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420726    5503 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bae499a-6abf-4f3a-8e0b-9bb821884f74-tmp\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:14.420721 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420740    5503 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-trusted-ca-bundle\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:14.421041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.420946    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config" pod "3bae499a-6abf-4f3a-8e0b-9bb821884f74" (UID: "3bae499a-6abf-4f3a-8e0b-9bb821884f74"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:20:14.422437 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.422410    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx" pod "3bae499a-6abf-4f3a-8e0b-9bb821884f74" (UID: "3bae499a-6abf-4f3a-8e0b-9bb821884f74"). InnerVolumeSpecName "kube-api-access-48zdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:20:14.422437 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.422418    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert" pod "3bae499a-6abf-4f3a-8e0b-9bb821884f74" (UID: "3bae499a-6abf-4f3a-8e0b-9bb821884f74"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:14.422560 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.422491    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config" pod "3bae499a-6abf-4f3a-8e0b-9bb821884f74" (UID: "3bae499a-6abf-4f3a-8e0b-9bb821884f74"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:20:14.521753 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.521732    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48zdx\" (UniqueName: \"kubernetes.io/projected/3bae499a-6abf-4f3a-8e0b-9bb821884f74-kube-api-access-48zdx\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:14.521753 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.521751    5503 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-serving-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:14.521901 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.521762    5503 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-oauth-config\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:14.521901 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.521771    5503 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-oauth-serving-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:14.521901 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:14.521780    5503 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3bae499a-6abf-4f3a-8e0b-9bb821884f74-console-config\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:20:15.103427 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.103401    5503 generic.go:356] "Generic (PLEG): container finished" podID="3bae499a-6abf-4f3a-8e0b-9bb821884f74" containerID="2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a" exitCode=0
Oct 30 22:20:15.103539 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.103440    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddcd679db-gbd7j" event={"ID":"3bae499a-6abf-4f3a-8e0b-9bb821884f74","Type":"ContainerDied","Data":"2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a"}
Oct 30 22:20:15.103539 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.103444    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7ddcd679db-gbd7j"
Oct 30 22:20:15.103539 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.103473    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7ddcd679db-gbd7j" event={"ID":"3bae499a-6abf-4f3a-8e0b-9bb821884f74","Type":"ContainerDied","Data":"ff634edbd2c1ec04cba1038ff5f6834e80222d86bcdecc21de57862cbb681415"}
Oct 30 22:20:15.103539 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.103492    5503 scope.go:122] "RemoveContainer" containerID="2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a"
Oct 30 22:20:15.112223 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.112203    5503 scope.go:122] "RemoveContainer" containerID="2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a"
Oct 30 22:20:15.112500 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:20:15.112481    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a\": container with ID starting with 2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a not found: ID does not exist" containerID="2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a"
Oct 30 22:20:15.112546 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.112509    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a"} err="failed to get container status \"2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a\": rpc error: code = NotFound desc = could not find container \"2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a\": container with ID starting with 2d52b3c985e9b1272c0260233def9c42de92266fa6bd994aabc4c5db7f20858a not found: ID does not exist"
Oct 30 22:20:15.125071 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.125052    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7ddcd679db-gbd7j"]
Oct 30 22:20:15.130342 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.130307    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7ddcd679db-gbd7j"]
Oct 30 22:20:15.387504 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:20:15.387450    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:20:15.856582 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:15.856557    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bae499a-6abf-4f3a-8e0b-9bb821884f74" path="/var/lib/kubelet/pods/3bae499a-6abf-4f3a-8e0b-9bb821884f74/volumes"
Oct 30 22:20:28.866917 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:28.866885    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:38.838575 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:38.838547    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:43.307155 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:43.307126    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:43.420220 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:20:43.420195    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:20:45.387405 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:20:45.387373    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:21:14.712980 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:21:14.712941    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:15.386983 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:21:15.386952    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:21:36.011629 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:21:36.011598    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:38.909874 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:21:38.909839    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:39.030110 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:21:39.030081    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:21:45.389031 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:21:45.388994    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:22:15.389336 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:22:15.389300    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:22:24.115043 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:24.114976    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:24.216628 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:24.216595    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:28.934363 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:28.934262    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:32.699741 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:32.699692    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:32.803564 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:32.803515    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:32.923102 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:32.923055    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:34.203010 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.202974    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"]
Oct 30 22:22:34.203386 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.203271    5503 cpu_manager.go:414] "containerMap: removing container" podUID="3bae499a-6abf-4f3a-8e0b-9bb821884f74" containerName="console"
Oct 30 22:22:34.203386 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.203282    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bae499a-6abf-4f3a-8e0b-9bb821884f74" containerName="console"
Oct 30 22:22:34.203386 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.203354    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bae499a-6abf-4f3a-8e0b-9bb821884f74" containerName="console"
Oct 30 22:22:34.206424 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.206402    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.211990 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.211967    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 22:22:34.212119 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.212045    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"98a21a31f7a8ae0212b9c22cc643f9202bfa37-6b211c7f-dockercfg-hddx4\""
Oct 30 22:22:34.212119 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.212055    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 22:22:34.227223 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.227194    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"]
Oct 30 22:22:34.312638 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.312604    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-bundle\") pod \"98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") " pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.312863 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.312652    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx2l2\" (UniqueName: \"kubernetes.io/projected/b28d63b7-a484-4410-9ad6-14ffdb4b757f-kube-api-access-nx2l2\") pod \"98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") " pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.312863 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.312821    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-util\") pod \"98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") " pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.413453 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.413418    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-util\") pod \"98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") " pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.413528 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.413470    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-bundle\") pod \"98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") " pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.413528 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.413505    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx2l2\" (UniqueName: \"kubernetes.io/projected/b28d63b7-a484-4410-9ad6-14ffdb4b757f-kube-api-access-nx2l2\") pod \"98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") " pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.413919 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.413901    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-util\") pod \"98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") " pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.413965 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.413953    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-bundle\") pod \"98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") " pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.426769 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.426742    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx2l2\" (UniqueName: \"kubernetes.io/projected/b28d63b7-a484-4410-9ad6-14ffdb4b757f-kube-api-access-nx2l2\") pod \"98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") " pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.521992 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.521900    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:34.796553 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:34.796479    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"]
Oct 30 22:22:34.799487 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:22:34.799461    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb28d63b7_a484_4410_9ad6_14ffdb4b757f.slice/crio-2e3f697316d1769c212ee3a63fa200d00fc7ad4db83c0b353d00008939ea3d23 WatchSource:0}: Error finding container 2e3f697316d1769c212ee3a63fa200d00fc7ad4db83c0b353d00008939ea3d23: Status 404 returned error can't find the container with id 2e3f697316d1769c212ee3a63fa200d00fc7ad4db83c0b353d00008939ea3d23
Oct 30 22:22:35.554879 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:35.554835    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl" event={"ID":"b28d63b7-a484-4410-9ad6-14ffdb4b757f","Type":"ContainerStarted","Data":"2e3f697316d1769c212ee3a63fa200d00fc7ad4db83c0b353d00008939ea3d23"}
Oct 30 22:22:38.942619 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:38.942544    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:41.578870 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:41.578831    5503 generic.go:356] "Generic (PLEG): container finished" podID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerID="7325102d831c51e834bd054890b7ca2257a30e5b8a633932c14247ee7f2a4330" exitCode=0
Oct 30 22:22:41.579276 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:41.578921    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl" event={"ID":"b28d63b7-a484-4410-9ad6-14ffdb4b757f","Type":"ContainerDied","Data":"7325102d831c51e834bd054890b7ca2257a30e5b8a633932c14247ee7f2a4330"}
Oct 30 22:22:44.591886 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:44.591854    5503 generic.go:356] "Generic (PLEG): container finished" podID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerID="899f529fb896f6506f114a8354633df869fefdca99b65ff036d192c80531cf8b" exitCode=0
Oct 30 22:22:44.592289 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:44.591896    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl" event={"ID":"b28d63b7-a484-4410-9ad6-14ffdb4b757f","Type":"ContainerDied","Data":"899f529fb896f6506f114a8354633df869fefdca99b65ff036d192c80531cf8b"}
Oct 30 22:22:45.395153 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:22:45.395111    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:22:50.618248 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:50.618211    5503 generic.go:356] "Generic (PLEG): container finished" podID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerID="cb91d5555c12e220b6c9853c4ed59979342ab32ec7fcbd976a97c7cd5578906b" exitCode=0
Oct 30 22:22:50.618835 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:50.618297    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl" event={"ID":"b28d63b7-a484-4410-9ad6-14ffdb4b757f","Type":"ContainerDied","Data":"cb91d5555c12e220b6c9853c4ed59979342ab32ec7fcbd976a97c7cd5578906b"}
Oct 30 22:22:51.640349 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.640309    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:51.739815 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.739784    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:51.745939 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.745915    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:51.866760 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.866728    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-util\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-util\") pod \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") "
Oct 30 22:22:51.866955 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.866780    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b28d63b7-a484-4410-9ad6-14ffdb4b757f-kube-api-access-nx2l2\" (UniqueName: \"kubernetes.io/projected/b28d63b7-a484-4410-9ad6-14ffdb4b757f-kube-api-access-nx2l2\") pod \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") "
Oct 30 22:22:51.866955 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.866813    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-bundle\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-bundle\") pod \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\" (UID: \"b28d63b7-a484-4410-9ad6-14ffdb4b757f\") "
Oct 30 22:22:51.867444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.867423    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-bundle" pod "b28d63b7-a484-4410-9ad6-14ffdb4b757f" (UID: "b28d63b7-a484-4410-9ad6-14ffdb4b757f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:51.869274 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.869249    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b28d63b7-a484-4410-9ad6-14ffdb4b757f-kube-api-access-nx2l2" pod "b28d63b7-a484-4410-9ad6-14ffdb4b757f" (UID: "b28d63b7-a484-4410-9ad6-14ffdb4b757f"). InnerVolumeSpecName "kube-api-access-nx2l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:22:51.873262 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.873208    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-util" pod "b28d63b7-a484-4410-9ad6-14ffdb4b757f" (UID: "b28d63b7-a484-4410-9ad6-14ffdb4b757f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:22:51.967319 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.967283    5503 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-util\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:51.967319 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.967315    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx2l2\" (UniqueName: \"kubernetes.io/projected/b28d63b7-a484-4410-9ad6-14ffdb4b757f-kube-api-access-nx2l2\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:51.967319 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:51.967326    5503 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b28d63b7-a484-4410-9ad6-14ffdb4b757f-bundle\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:22:52.626443 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:52.626414    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl"
Oct 30 22:22:52.626639 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:52.626413    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/98a21a31f7a8ae0212b9c22cc643f9202bfa37cdda9b658bc303ea820cmk7kl" event={"ID":"b28d63b7-a484-4410-9ad6-14ffdb4b757f","Type":"ContainerDied","Data":"2e3f697316d1769c212ee3a63fa200d00fc7ad4db83c0b353d00008939ea3d23"}
Oct 30 22:22:52.626639 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:52.626530    5503 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e3f697316d1769c212ee3a63fa200d00fc7ad4db83c0b353d00008939ea3d23"
Oct 30 22:22:56.150117 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:56.150079    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:56.248286 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:56.248250    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:22:58.200869 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.200832    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"]
Oct 30 22:22:58.201272 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.201123    5503 cpu_manager.go:414] "containerMap: removing container" podUID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerName="pull"
Oct 30 22:22:58.201272 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.201133    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerName="pull"
Oct 30 22:22:58.201272 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.201149    5503 cpu_manager.go:414] "containerMap: removing container" podUID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerName="util"
Oct 30 22:22:58.201272 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.201154    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerName="util"
Oct 30 22:22:58.201272 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.201161    5503 cpu_manager.go:414] "containerMap: removing container" podUID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerName="extract"
Oct 30 22:22:58.201272 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.201166    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerName="extract"
Oct 30 22:22:58.201272 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.201204    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="b28d63b7-a484-4410-9ad6-14ffdb4b757f" containerName="extract"
Oct 30 22:22:58.203981 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.203965    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:22:58.210229 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.210175    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Oct 30 22:22:58.210307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.210274    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Oct 30 22:22:58.210968 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.210945    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Oct 30 22:22:58.211067 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.210979    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-pxdjf\""
Oct 30 22:22:58.218142 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.218109    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"]
Oct 30 22:22:58.322410 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.322370    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-456hk\" (UniqueName: \"kubernetes.io/projected/36819a6e-743e-4a63-8054-d05711b3c3f5-kube-api-access-456hk\") pod \"custom-metrics-autoscaler-operator-c667576c8-dxs4g\" (UID: \"36819a6e-743e-4a63-8054-d05711b3c3f5\") " pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:22:58.322598 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.322488    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/36819a6e-743e-4a63-8054-d05711b3c3f5-certificates\") pod \"custom-metrics-autoscaler-operator-c667576c8-dxs4g\" (UID: \"36819a6e-743e-4a63-8054-d05711b3c3f5\") " pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:22:58.423741 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.423710    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/36819a6e-743e-4a63-8054-d05711b3c3f5-certificates\") pod \"custom-metrics-autoscaler-operator-c667576c8-dxs4g\" (UID: \"36819a6e-743e-4a63-8054-d05711b3c3f5\") " pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:22:58.423885 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.423776    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-456hk\" (UniqueName: \"kubernetes.io/projected/36819a6e-743e-4a63-8054-d05711b3c3f5-kube-api-access-456hk\") pod \"custom-metrics-autoscaler-operator-c667576c8-dxs4g\" (UID: \"36819a6e-743e-4a63-8054-d05711b3c3f5\") " pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:22:58.426471 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.426444    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/36819a6e-743e-4a63-8054-d05711b3c3f5-certificates\") pod \"custom-metrics-autoscaler-operator-c667576c8-dxs4g\" (UID: \"36819a6e-743e-4a63-8054-d05711b3c3f5\") " pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:22:58.437286 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.437251    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-456hk\" (UniqueName: \"kubernetes.io/projected/36819a6e-743e-4a63-8054-d05711b3c3f5-kube-api-access-456hk\") pod \"custom-metrics-autoscaler-operator-c667576c8-dxs4g\" (UID: \"36819a6e-743e-4a63-8054-d05711b3c3f5\") " pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:22:58.516284 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.516202    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:22:58.795228 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:58.795126    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"]
Oct 30 22:22:58.799094 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:22:58.799067    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36819a6e_743e_4a63_8054_d05711b3c3f5.slice/crio-0bb18deeddec42d6344b44079e35a8cbc286d8f2ed6921c62e239167bde25dc8 WatchSource:0}: Error finding container 0bb18deeddec42d6344b44079e35a8cbc286d8f2ed6921c62e239167bde25dc8: Status 404 returned error can't find the container with id 0bb18deeddec42d6344b44079e35a8cbc286d8f2ed6921c62e239167bde25dc8
Oct 30 22:22:59.653645 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:22:59.653609    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g" event={"ID":"36819a6e-743e-4a63-8054-d05711b3c3f5","Type":"ContainerStarted","Data":"0bb18deeddec42d6344b44079e35a8cbc286d8f2ed6921c62e239167bde25dc8"}
Oct 30 22:23:03.361120 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.361075    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:03.656609 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.656532    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-7dc85965f-g2fhk"]
Oct 30 22:23:03.659739 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.659720    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:03.666299 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.666274    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Oct 30 22:23:03.666417 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.666365    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\""
Oct 30 22:23:03.666417 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.666404    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-httbp\""
Oct 30 22:23:03.671626 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.671598    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g" event={"ID":"36819a6e-743e-4a63-8054-d05711b3c3f5","Type":"ContainerStarted","Data":"5e74330f0fb9b93cc8067916439d6d661e958c73366112be25fba2240a939585"}
Oct 30 22:23:03.671752 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.671713    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:23:03.677973 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.677952    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-7dc85965f-g2fhk"]
Oct 30 22:23:03.686741 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.686713    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:03.722233 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.722158    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g" podStartSLOduration=1.605718769 podStartE2EDuration="5.722140535s" totalImagesPullingTime="4.116421766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:03.721635769 +0000 UTC m=+225.990717302" watchObservedRunningTime="2026-10-30 22:23:03.722140535 +0000 UTC m=+225.991222058"
Oct 30 22:23:03.769132 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.769094    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-cabundle0\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:03.769295 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.769139    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkjln\" (UniqueName: \"kubernetes.io/projected/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-kube-api-access-lkjln\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:03.769295 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.769278    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-certificates\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:03.870498 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.870457    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-cabundle0\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:03.870498 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.870495    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkjln\" (UniqueName: \"kubernetes.io/projected/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-kube-api-access-lkjln\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:03.870793 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.870529    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-certificates\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:03.870793 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:23:03.870638    5503 secret.go:281] references non-existent secret key: ca.crt
Oct 30 22:23:03.870793 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:23:03.870651    5503 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt
Oct 30 22:23:03.870793 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:23:03.870659    5503 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-7dc85965f-g2fhk: references non-existent secret key: ca.crt
Oct 30 22:23:03.870793 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:23:03.870725    5503 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-certificates podName:041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49 nodeName:}" failed. No retries permitted until 2026-10-30 22:23:04.370710761 +0000 UTC m=+226.639792274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-certificates") pod "keda-operator-7dc85965f-g2fhk" (UID: "041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49") : references non-existent secret key: ca.crt
Oct 30 22:23:03.871232 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.871207    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-cabundle0\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:03.882191 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.882151    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:03.884597 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.884572    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkjln\" (UniqueName: \"kubernetes.io/projected/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-kube-api-access-lkjln\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:03.960215 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.960131    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"]
Oct 30 22:23:03.963359 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.963341    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:03.967711 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.967686    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Oct 30 22:23:03.995719 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:03.995686    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:04.012004 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.011975    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"]
Oct 30 22:23:04.073179 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.073143    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/67f6f60b-7388-478d-bc5b-75cbcfbd191a-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-8k48v\" (UID: \"67f6f60b-7388-478d-bc5b-75cbcfbd191a\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.073358 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.073279    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzrwm\" (UniqueName: \"kubernetes.io/projected/67f6f60b-7388-478d-bc5b-75cbcfbd191a-kube-api-access-wzrwm\") pod \"keda-metrics-apiserver-86bd856cfb-8k48v\" (UID: \"67f6f60b-7388-478d-bc5b-75cbcfbd191a\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.073358 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.073325    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/67f6f60b-7388-478d-bc5b-75cbcfbd191a-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-8k48v\" (UID: \"67f6f60b-7388-478d-bc5b-75cbcfbd191a\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.174821 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.174785    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzrwm\" (UniqueName: \"kubernetes.io/projected/67f6f60b-7388-478d-bc5b-75cbcfbd191a-kube-api-access-wzrwm\") pod \"keda-metrics-apiserver-86bd856cfb-8k48v\" (UID: \"67f6f60b-7388-478d-bc5b-75cbcfbd191a\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.175007 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.174862    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/67f6f60b-7388-478d-bc5b-75cbcfbd191a-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-8k48v\" (UID: \"67f6f60b-7388-478d-bc5b-75cbcfbd191a\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.175007 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.174901    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/67f6f60b-7388-478d-bc5b-75cbcfbd191a-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-8k48v\" (UID: \"67f6f60b-7388-478d-bc5b-75cbcfbd191a\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.175351 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.175332    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/67f6f60b-7388-478d-bc5b-75cbcfbd191a-temp-vol\") pod \"keda-metrics-apiserver-86bd856cfb-8k48v\" (UID: \"67f6f60b-7388-478d-bc5b-75cbcfbd191a\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.177551 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.177526    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/67f6f60b-7388-478d-bc5b-75cbcfbd191a-certificates\") pod \"keda-metrics-apiserver-86bd856cfb-8k48v\" (UID: \"67f6f60b-7388-478d-bc5b-75cbcfbd191a\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.188447 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.188418    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:04.188643 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.188623    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzrwm\" (UniqueName: \"kubernetes.io/projected/67f6f60b-7388-478d-bc5b-75cbcfbd191a-kube-api-access-wzrwm\") pod \"keda-metrics-apiserver-86bd856cfb-8k48v\" (UID: \"67f6f60b-7388-478d-bc5b-75cbcfbd191a\") " pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.219933 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.219852    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-r69ft"]
Oct 30 22:23:04.223328 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.223305    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:04.231430 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.231407    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Oct 30 22:23:04.235972 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.235944    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-r69ft"]
Oct 30 22:23:04.275934 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.275896    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:04.295994 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.295966    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:04.377297 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.377255    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/644987b9-d739-4550-a9f0-70b982577700-certificates\") pod \"keda-admission-7b7c985c58-r69ft\" (UID: \"644987b9-d739-4550-a9f0-70b982577700\") " pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:04.377742 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.377303    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnv5k\" (UniqueName: \"kubernetes.io/projected/644987b9-d739-4550-a9f0-70b982577700-kube-api-access-mnv5k\") pod \"keda-admission-7b7c985c58-r69ft\" (UID: \"644987b9-d739-4550-a9f0-70b982577700\") " pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:04.377742 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.377506    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-certificates\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:04.381527 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.381500    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49-certificates\") pod \"keda-operator-7dc85965f-g2fhk\" (UID: \"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49\") " pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:04.409820 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.409788    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:04.479707 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.479583    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/644987b9-d739-4550-a9f0-70b982577700-certificates\") pod \"keda-admission-7b7c985c58-r69ft\" (UID: \"644987b9-d739-4550-a9f0-70b982577700\") " pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:04.479707 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.479640    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnv5k\" (UniqueName: \"kubernetes.io/projected/644987b9-d739-4550-a9f0-70b982577700-kube-api-access-mnv5k\") pod \"keda-admission-7b7c985c58-r69ft\" (UID: \"644987b9-d739-4550-a9f0-70b982577700\") " pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:04.482265 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.482236    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/644987b9-d739-4550-a9f0-70b982577700-certificates\") pod \"keda-admission-7b7c985c58-r69ft\" (UID: \"644987b9-d739-4550-a9f0-70b982577700\") " pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:04.493321 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.493293    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnv5k\" (UniqueName: \"kubernetes.io/projected/644987b9-d739-4550-a9f0-70b982577700-kube-api-access-mnv5k\") pod \"keda-admission-7b7c985c58-r69ft\" (UID: \"644987b9-d739-4550-a9f0-70b982577700\") " pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:04.536791 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.536749    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:04.564938 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.564904    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"]
Oct 30 22:23:04.570174 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:23:04.570139    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67f6f60b_7388_478d_bc5b_75cbcfbd191a.slice/crio-8dbca637e88e28f6768daf84f255b939265a2e0c9eaf33b9cf335d1b5bdacbea WatchSource:0}: Error finding container 8dbca637e88e28f6768daf84f255b939265a2e0c9eaf33b9cf335d1b5bdacbea: Status 404 returned error can't find the container with id 8dbca637e88e28f6768daf84f255b939265a2e0c9eaf33b9cf335d1b5bdacbea
Oct 30 22:23:04.574700 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.571889    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:04.682353 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.682301    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v" event={"ID":"67f6f60b-7388-478d-bc5b-75cbcfbd191a","Type":"ContainerStarted","Data":"8dbca637e88e28f6768daf84f255b939265a2e0c9eaf33b9cf335d1b5bdacbea"}
Oct 30 22:23:04.832751 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.832647    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-7b7c985c58-r69ft"]
Oct 30 22:23:04.837274 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:23:04.837130    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod644987b9_d739_4550_a9f0_70b982577700.slice/crio-72f4cc8bbb0fa7b016edc5ec52cc38db2d4a923e4d18e602a162761bfcda62f9 WatchSource:0}: Error finding container 72f4cc8bbb0fa7b016edc5ec52cc38db2d4a923e4d18e602a162761bfcda62f9: Status 404 returned error can't find the container with id 72f4cc8bbb0fa7b016edc5ec52cc38db2d4a923e4d18e602a162761bfcda62f9
Oct 30 22:23:04.872641 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:04.872608    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-7dc85965f-g2fhk"]
Oct 30 22:23:04.876100 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:23:04.876071    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041dcfdc_0d3b_40a7_8eb6_9fcabe7cce49.slice/crio-a0e781016674182d194b4cafa4e5ebdc56501625e53fe24c31e1b383eb5873e2 WatchSource:0}: Error finding container a0e781016674182d194b4cafa4e5ebdc56501625e53fe24c31e1b383eb5873e2: Status 404 returned error can't find the container with id a0e781016674182d194b4cafa4e5ebdc56501625e53fe24c31e1b383eb5873e2
Oct 30 22:23:05.008842 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:05.008806    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:05.229952 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:05.229905    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:05.342883 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:05.342836    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:05.687163 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:05.687015    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-r69ft" event={"ID":"644987b9-d739-4550-a9f0-70b982577700","Type":"ContainerStarted","Data":"72f4cc8bbb0fa7b016edc5ec52cc38db2d4a923e4d18e602a162761bfcda62f9"}
Oct 30 22:23:05.706130 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:05.688160    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-7dc85965f-g2fhk" event={"ID":"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49","Type":"ContainerStarted","Data":"a0e781016674182d194b4cafa4e5ebdc56501625e53fe24c31e1b383eb5873e2"}
Oct 30 22:23:07.699119 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:07.699079    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-7b7c985c58-r69ft" event={"ID":"644987b9-d739-4550-a9f0-70b982577700","Type":"ContainerStarted","Data":"b29895b0be5672d36ad95ecd010472edad26a90d9db55c9b2984469f832406b1"}
Oct 30 22:23:07.699537 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:07.699200    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:07.720569 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:07.720381    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-admission-7b7c985c58-r69ft" podStartSLOduration=1.682975382 podStartE2EDuration="3.720361759s" totalImagesPullingTime="2.037386377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:07.719955545 +0000 UTC m=+229.989037068" watchObservedRunningTime="2026-10-30 22:23:07.720361759 +0000 UTC m=+229.989443283"
Oct 30 22:23:08.705539 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:08.705502    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v" event={"ID":"67f6f60b-7388-478d-bc5b-75cbcfbd191a","Type":"ContainerStarted","Data":"dfd332403bc56b5aa290d3fa7ac8afeae411f556e48ab74d83e02eeca78e63be"}
Oct 30 22:23:08.706034 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:08.705594    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:08.726174 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:08.726116    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v" podStartSLOduration=2.374898585 podStartE2EDuration="5.726102462s" totalImagesPullingTime="3.351203877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:08.72591369 +0000 UTC m=+230.994995213" watchObservedRunningTime="2026-10-30 22:23:08.726102462 +0000 UTC m=+230.995183985"
Oct 30 22:23:10.716561 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:10.716526    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-7dc85965f-g2fhk" event={"ID":"041dcfdc-0d3b-40a7-8eb6-9fcabe7cce49","Type":"ContainerStarted","Data":"8651bbb01e526db057f1380d86fc62cca4ff1d52fec09afd9a404a986983f3a4"}
Oct 30 22:23:10.717022 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:10.716704    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:10.736899 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:10.736847    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-keda/keda-operator-7dc85965f-g2fhk" podStartSLOduration=2.916526457 podStartE2EDuration="7.736831787s" totalImagesPullingTime="4.82030533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:23:10.736063243 +0000 UTC m=+233.005144760" watchObservedRunningTime="2026-10-30 22:23:10.736831787 +0000 UTC m=+233.005913302"
Oct 30 22:23:15.390962 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:23:15.390923    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:23:19.714880 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:19.714850    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-86bd856cfb-8k48v"
Oct 30 22:23:24.684808 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:24.684774    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-c667576c8-dxs4g"
Oct 30 22:23:28.708715 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:28.708660    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-7b7c985c58-r69ft"
Oct 30 22:23:31.723333 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:31.723305    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-7dc85965f-g2fhk"
Oct 30 22:23:39.019691 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:39.019631    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:45.389133 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:23:45.389090    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:23:48.896304 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:48.896270    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:49.000035 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:49.000004    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:56.504571 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.504536    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:56.615653 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.615618    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:23:56.844741 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.844652    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"]
Oct 30 22:23:56.847196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.847173    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:56.856277 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.856252    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 22:23:56.857251 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.857237    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 22:23:56.858869 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.858850    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-glchb\""
Oct 30 22:23:56.859442 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.859425    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"]
Oct 30 22:23:56.945561 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.945528    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:56.945766 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.945575    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clwb6\" (UniqueName: \"kubernetes.io/projected/658eb256-3e3b-4ab0-8e81-b82a2d692463-kube-api-access-clwb6\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:56.945766 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:56.945708    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:57.046520 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.046481    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:57.046729 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.046531    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clwb6\" (UniqueName: \"kubernetes.io/projected/658eb256-3e3b-4ab0-8e81-b82a2d692463-kube-api-access-clwb6\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:57.046729 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.046581    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:57.046970 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.046954    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:57.047015 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.046993    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:57.061809 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.061775    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clwb6\" (UniqueName: \"kubernetes.io/projected/658eb256-3e3b-4ab0-8e81-b82a2d692463-kube-api-access-clwb6\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:57.160024 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.159927    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:23:57.435116 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.435037    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"]
Oct 30 22:23:57.438122 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:23:57.438094    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod658eb256_3e3b_4ab0_8e81_b82a2d692463.slice/crio-0035a2c4bef3f6d937fbd8c5e5d90bbe45b492be9d19f1110f86ea2cccb8b839 WatchSource:0}: Error finding container 0035a2c4bef3f6d937fbd8c5e5d90bbe45b492be9d19f1110f86ea2cccb8b839: Status 404 returned error can't find the container with id 0035a2c4bef3f6d937fbd8c5e5d90bbe45b492be9d19f1110f86ea2cccb8b839
Oct 30 22:23:57.909367 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.909328    5503 generic.go:356] "Generic (PLEG): container finished" podID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerID="230a1e85c194e829f8e50be90bff96d76f9cc6cd1aa0196ee519625400105047" exitCode=0
Oct 30 22:23:57.909770 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.909411    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q" event={"ID":"658eb256-3e3b-4ab0-8e81-b82a2d692463","Type":"ContainerDied","Data":"230a1e85c194e829f8e50be90bff96d76f9cc6cd1aa0196ee519625400105047"}
Oct 30 22:23:57.909770 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:23:57.909464    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q" event={"ID":"658eb256-3e3b-4ab0-8e81-b82a2d692463","Type":"ContainerStarted","Data":"0035a2c4bef3f6d937fbd8c5e5d90bbe45b492be9d19f1110f86ea2cccb8b839"}
Oct 30 22:24:00.924122 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:00.924084    5503 generic.go:356] "Generic (PLEG): container finished" podID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerID="9ee9be4bc9c0a632bfa7e8ac85aae176620a5b3000e5d4cc3d96a66cf94fb1b4" exitCode=0
Oct 30 22:24:00.924497 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:00.924135    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q" event={"ID":"658eb256-3e3b-4ab0-8e81-b82a2d692463","Type":"ContainerDied","Data":"9ee9be4bc9c0a632bfa7e8ac85aae176620a5b3000e5d4cc3d96a66cf94fb1b4"}
Oct 30 22:24:01.929926 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:01.929894    5503 generic.go:356] "Generic (PLEG): container finished" podID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerID="c968d7717c6630224b9fa92bbd964beb3701ffadac49d999a51d3cb54f6e16c2" exitCode=0
Oct 30 22:24:01.930348 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:01.929947    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q" event={"ID":"658eb256-3e3b-4ab0-8e81-b82a2d692463","Type":"ContainerDied","Data":"c968d7717c6630224b9fa92bbd964beb3701ffadac49d999a51d3cb54f6e16c2"}
Oct 30 22:24:03.060405 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.060373    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:24:03.097729 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.097695    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-util\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-util\") pod \"658eb256-3e3b-4ab0-8e81-b82a2d692463\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") "
Oct 30 22:24:03.097928 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.097801    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-bundle\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-bundle\") pod \"658eb256-3e3b-4ab0-8e81-b82a2d692463\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") "
Oct 30 22:24:03.097928 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.097862    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/658eb256-3e3b-4ab0-8e81-b82a2d692463-kube-api-access-clwb6\" (UniqueName: \"kubernetes.io/projected/658eb256-3e3b-4ab0-8e81-b82a2d692463-kube-api-access-clwb6\") pod \"658eb256-3e3b-4ab0-8e81-b82a2d692463\" (UID: \"658eb256-3e3b-4ab0-8e81-b82a2d692463\") "
Oct 30 22:24:03.098424 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.098395    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-bundle" pod "658eb256-3e3b-4ab0-8e81-b82a2d692463" (UID: "658eb256-3e3b-4ab0-8e81-b82a2d692463"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:03.099996 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.099976    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/658eb256-3e3b-4ab0-8e81-b82a2d692463-kube-api-access-clwb6" pod "658eb256-3e3b-4ab0-8e81-b82a2d692463" (UID: "658eb256-3e3b-4ab0-8e81-b82a2d692463"). InnerVolumeSpecName "kube-api-access-clwb6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:03.102089 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.102040    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-util" pod "658eb256-3e3b-4ab0-8e81-b82a2d692463" (UID: "658eb256-3e3b-4ab0-8e81-b82a2d692463"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:03.198817 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.198719    5503 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-bundle\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:03.198817 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.198767    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clwb6\" (UniqueName: \"kubernetes.io/projected/658eb256-3e3b-4ab0-8e81-b82a2d692463-kube-api-access-clwb6\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:03.198817 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.198777    5503 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/658eb256-3e3b-4ab0-8e81-b82a2d692463-util\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:03.939816 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.939789    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q"
Oct 30 22:24:03.939976 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.939781    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651c5t75q" event={"ID":"658eb256-3e3b-4ab0-8e81-b82a2d692463","Type":"ContainerDied","Data":"0035a2c4bef3f6d937fbd8c5e5d90bbe45b492be9d19f1110f86ea2cccb8b839"}
Oct 30 22:24:03.939976 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:03.939932    5503 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0035a2c4bef3f6d937fbd8c5e5d90bbe45b492be9d19f1110f86ea2cccb8b839"
Oct 30 22:24:08.043144 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:08.043045    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:08.283490 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:08.283460    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:10.262311 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.262270    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"]
Oct 30 22:24:10.262747 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.262732    5503 cpu_manager.go:414] "containerMap: removing container" podUID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerName="util"
Oct 30 22:24:10.262804 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.262749    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerName="util"
Oct 30 22:24:10.262804 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.262766    5503 cpu_manager.go:414] "containerMap: removing container" podUID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerName="extract"
Oct 30 22:24:10.262804 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.262771    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerName="extract"
Oct 30 22:24:10.262804 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.262785    5503 cpu_manager.go:414] "containerMap: removing container" podUID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerName="pull"
Oct 30 22:24:10.262804 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.262790    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerName="pull"
Oct 30 22:24:10.262944 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.262863    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="658eb256-3e3b-4ab0-8e81-b82a2d692463" containerName="extract"
Oct 30 22:24:10.266782 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.266767    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"
Oct 30 22:24:10.275717 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.275694    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-hmcsf\""
Oct 30 22:24:10.276066 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.276037    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 22:24:10.277921 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.277906    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:24:10.280157 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.280132    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"]
Oct 30 22:24:10.362986 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.362947    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt2w6\" (UniqueName: \"kubernetes.io/projected/41f9406e-d78a-4439-b115-56d952fc008e-kube-api-access-bt2w6\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr\" (UID: \"41f9406e-d78a-4439-b115-56d952fc008e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"
Oct 30 22:24:10.362986 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.362991    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41f9406e-d78a-4439-b115-56d952fc008e-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr\" (UID: \"41f9406e-d78a-4439-b115-56d952fc008e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"
Oct 30 22:24:10.463998 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.463959    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt2w6\" (UniqueName: \"kubernetes.io/projected/41f9406e-d78a-4439-b115-56d952fc008e-kube-api-access-bt2w6\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr\" (UID: \"41f9406e-d78a-4439-b115-56d952fc008e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"
Oct 30 22:24:10.463998 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.464002    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41f9406e-d78a-4439-b115-56d952fc008e-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr\" (UID: \"41f9406e-d78a-4439-b115-56d952fc008e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"
Oct 30 22:24:10.464461 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.464435    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41f9406e-d78a-4439-b115-56d952fc008e-tmp\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr\" (UID: \"41f9406e-d78a-4439-b115-56d952fc008e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"
Oct 30 22:24:10.477579 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.477551    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt2w6\" (UniqueName: \"kubernetes.io/projected/41f9406e-d78a-4439-b115-56d952fc008e-kube-api-access-bt2w6\") pod \"cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr\" (UID: \"41f9406e-d78a-4439-b115-56d952fc008e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"
Oct 30 22:24:10.578944 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.578857    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"
Oct 30 22:24:10.866253 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.866141    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr"]
Oct 30 22:24:10.869426 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:24:10.869400    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f9406e_d78a_4439_b115_56d952fc008e.slice/crio-059987b7409b8480372cf408b98c445af381aacc6a6ae859a9d78d6f75a5b598 WatchSource:0}: Error finding container 059987b7409b8480372cf408b98c445af381aacc6a6ae859a9d78d6f75a5b598: Status 404 returned error can't find the container with id 059987b7409b8480372cf408b98c445af381aacc6a6ae859a9d78d6f75a5b598
Oct 30 22:24:10.969030 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:10.968996    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr" event={"ID":"41f9406e-d78a-4439-b115-56d952fc008e","Type":"ContainerStarted","Data":"059987b7409b8480372cf408b98c445af381aacc6a6ae859a9d78d6f75a5b598"}
Oct 30 22:24:11.657138 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:11.657104    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:11.862481 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:11.862449    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:13.985717 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:13.985618    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr" event={"ID":"41f9406e-d78a-4439-b115-56d952fc008e","Type":"ContainerStarted","Data":"cfa51a680c46e0199b30f89629f75cc9a2b7b64de076b2bcdb7e0b309f0c79bd"}
Oct 30 22:24:14.016944 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:14.016890    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-7d87bcd5fd-5wsxr" podStartSLOduration=1.26364206 podStartE2EDuration="4.016871607s" totalImagesPullingTime="2.753229547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:14.015177242 +0000 UTC m=+296.284258801" watchObservedRunningTime="2026-10-30 22:24:14.016871607 +0000 UTC m=+296.285953129"
Oct 30 22:24:14.529813 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:14.529774    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:14.895571 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:14.895533    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:15.390400 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:24:15.390363    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:24:16.425290 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:16.425249    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:16.534086 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:16.534021    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:16.875142 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:16.875112    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:17.049587 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:17.049556    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:18.861477 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:18.861439    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:18.970501 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:18.970470    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:19.079865 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.079834    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:19.099059 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.099030    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"]
Oct 30 22:24:19.102590 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.102572    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.107698 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.107648    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 22:24:19.107883 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.107750    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 22:24:19.107883 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.107813    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"61b1ff68c7255ee5c5329fe19a5d96293a9578-ce91a85d-dockercfg-4zrgq\""
Oct 30 22:24:19.113715 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.113632    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"]
Oct 30 22:24:19.243993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.243951    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.244206 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.244007    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.244206 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.244106    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lstzm\" (UniqueName: \"kubernetes.io/projected/c3682df0-6049-475e-a9b6-2ce93412d5ef-kube-api-access-lstzm\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.344924 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.344879    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.345127 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.344949    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.345127 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.344974    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lstzm\" (UniqueName: \"kubernetes.io/projected/c3682df0-6049-475e-a9b6-2ce93412d5ef-kube-api-access-lstzm\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.345322 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.345295    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.345385 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.345331    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.360642 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.360611    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lstzm\" (UniqueName: \"kubernetes.io/projected/c3682df0-6049-475e-a9b6-2ce93412d5ef-kube-api-access-lstzm\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.419007 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.418933    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:19.693831 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:19.693805    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"]
Oct 30 22:24:19.695748 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:24:19.695723    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3682df0_6049_475e_a9b6_2ce93412d5ef.slice/crio-82a40603951e1831f34b12cf9a6133df97b22385f99af4955b0f35b154a09f90 WatchSource:0}: Error finding container 82a40603951e1831f34b12cf9a6133df97b22385f99af4955b0f35b154a09f90: Status 404 returned error can't find the container with id 82a40603951e1831f34b12cf9a6133df97b22385f99af4955b0f35b154a09f90
Oct 30 22:24:20.011949 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:20.011866    5503 generic.go:356] "Generic (PLEG): container finished" podID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerID="ba2a16878bdc44f4492d1a318bc936bc795b18eea08e8112547c01cbb2ba98bf" exitCode=0
Oct 30 22:24:20.012382 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:20.011952    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz" event={"ID":"c3682df0-6049-475e-a9b6-2ce93412d5ef","Type":"ContainerDied","Data":"ba2a16878bdc44f4492d1a318bc936bc795b18eea08e8112547c01cbb2ba98bf"}
Oct 30 22:24:20.012382 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:20.012003    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz" event={"ID":"c3682df0-6049-475e-a9b6-2ce93412d5ef","Type":"ContainerStarted","Data":"82a40603951e1831f34b12cf9a6133df97b22385f99af4955b0f35b154a09f90"}
Oct 30 22:24:23.025691 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:23.025640    5503 generic.go:356] "Generic (PLEG): container finished" podID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerID="b052bc26788ef3ed328f295d06885bf72530f2cbd49dfeba700554579c095d44" exitCode=0
Oct 30 22:24:23.026171 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:23.025721    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz" event={"ID":"c3682df0-6049-475e-a9b6-2ce93412d5ef","Type":"ContainerDied","Data":"b052bc26788ef3ed328f295d06885bf72530f2cbd49dfeba700554579c095d44"}
Oct 30 22:24:24.032410 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.032375    5503 generic.go:356] "Generic (PLEG): container finished" podID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerID="c01c011ce5469407a010a84708e6a4ac85754ec84e4b5f5c29aad27499d56074" exitCode=0
Oct 30 22:24:24.032868 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.032454    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz" event={"ID":"c3682df0-6049-475e-a9b6-2ce93412d5ef","Type":"ContainerDied","Data":"c01c011ce5469407a010a84708e6a4ac85754ec84e4b5f5c29aad27499d56074"}
Oct 30 22:24:24.105590 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.105555    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:24.918731 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.918695    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-n8hnc"]
Oct 30 22:24:24.922023 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.922005    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-n8hnc"
Oct 30 22:24:24.932790 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.932757    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 22:24:24.932790 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.932781    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-nd4l6\""
Oct 30 22:24:24.932972 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.932819    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 22:24:24.933460 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.933443    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-n8hnc"]
Oct 30 22:24:24.996014 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.995979    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czc45\" (UniqueName: \"kubernetes.io/projected/7c30467d-1b2e-4ffe-93c9-6eae9212bcee-kube-api-access-czc45\") pod \"cert-manager-597b95cb84-n8hnc\" (UID: \"7c30467d-1b2e-4ffe-93c9-6eae9212bcee\") " pod="cert-manager/cert-manager-597b95cb84-n8hnc"
Oct 30 22:24:24.996287 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:24.996271    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c30467d-1b2e-4ffe-93c9-6eae9212bcee-bound-sa-token\") pod \"cert-manager-597b95cb84-n8hnc\" (UID: \"7c30467d-1b2e-4ffe-93c9-6eae9212bcee\") " pod="cert-manager/cert-manager-597b95cb84-n8hnc"
Oct 30 22:24:25.096993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.096949    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czc45\" (UniqueName: \"kubernetes.io/projected/7c30467d-1b2e-4ffe-93c9-6eae9212bcee-kube-api-access-czc45\") pod \"cert-manager-597b95cb84-n8hnc\" (UID: \"7c30467d-1b2e-4ffe-93c9-6eae9212bcee\") " pod="cert-manager/cert-manager-597b95cb84-n8hnc"
Oct 30 22:24:25.097429 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.097045    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c30467d-1b2e-4ffe-93c9-6eae9212bcee-bound-sa-token\") pod \"cert-manager-597b95cb84-n8hnc\" (UID: \"7c30467d-1b2e-4ffe-93c9-6eae9212bcee\") " pod="cert-manager/cert-manager-597b95cb84-n8hnc"
Oct 30 22:24:25.110358 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.110326    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7c30467d-1b2e-4ffe-93c9-6eae9212bcee-bound-sa-token\") pod \"cert-manager-597b95cb84-n8hnc\" (UID: \"7c30467d-1b2e-4ffe-93c9-6eae9212bcee\") " pod="cert-manager/cert-manager-597b95cb84-n8hnc"
Oct 30 22:24:25.110493 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.110408    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czc45\" (UniqueName: \"kubernetes.io/projected/7c30467d-1b2e-4ffe-93c9-6eae9212bcee-kube-api-access-czc45\") pod \"cert-manager-597b95cb84-n8hnc\" (UID: \"7c30467d-1b2e-4ffe-93c9-6eae9212bcee\") " pod="cert-manager/cert-manager-597b95cb84-n8hnc"
Oct 30 22:24:25.164182 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.164157    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:25.245119 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.245020    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-n8hnc"
Oct 30 22:24:25.298830 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.298798    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-util\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-util\") pod \"c3682df0-6049-475e-a9b6-2ce93412d5ef\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") "
Oct 30 22:24:25.298950 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.298895    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c3682df0-6049-475e-a9b6-2ce93412d5ef-kube-api-access-lstzm\" (UniqueName: \"kubernetes.io/projected/c3682df0-6049-475e-a9b6-2ce93412d5ef-kube-api-access-lstzm\") pod \"c3682df0-6049-475e-a9b6-2ce93412d5ef\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") "
Oct 30 22:24:25.298992 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.298949    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-bundle\") pod \"c3682df0-6049-475e-a9b6-2ce93412d5ef\" (UID: \"c3682df0-6049-475e-a9b6-2ce93412d5ef\") "
Oct 30 22:24:25.299801 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.299770    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-bundle" pod "c3682df0-6049-475e-a9b6-2ce93412d5ef" (UID: "c3682df0-6049-475e-a9b6-2ce93412d5ef"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:25.304374 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.304342    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3682df0-6049-475e-a9b6-2ce93412d5ef-kube-api-access-lstzm" pod "c3682df0-6049-475e-a9b6-2ce93412d5ef" (UID: "c3682df0-6049-475e-a9b6-2ce93412d5ef"). InnerVolumeSpecName "kube-api-access-lstzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:24:25.305372 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.305338    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-util" pod "c3682df0-6049-475e-a9b6-2ce93412d5ef" (UID: "c3682df0-6049-475e-a9b6-2ce93412d5ef"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:24:25.400131 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.400069    5503 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-util\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:25.400131 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.400120    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lstzm\" (UniqueName: \"kubernetes.io/projected/c3682df0-6049-475e-a9b6-2ce93412d5ef-kube-api-access-lstzm\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:25.400131 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.400131    5503 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3682df0-6049-475e-a9b6-2ce93412d5ef-bundle\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:24:25.521255 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:25.521134    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-n8hnc"]
Oct 30 22:24:25.524004 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:24:25.523972    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c30467d_1b2e_4ffe_93c9_6eae9212bcee.slice/crio-44f3437198c54258ef5b9855a702be78e51fd4d2ab7d24877ca731aa88cea879 WatchSource:0}: Error finding container 44f3437198c54258ef5b9855a702be78e51fd4d2ab7d24877ca731aa88cea879: Status 404 returned error can't find the container with id 44f3437198c54258ef5b9855a702be78e51fd4d2ab7d24877ca731aa88cea879
Oct 30 22:24:26.041547 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:26.041510    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-n8hnc" event={"ID":"7c30467d-1b2e-4ffe-93c9-6eae9212bcee","Type":"ContainerStarted","Data":"44f3437198c54258ef5b9855a702be78e51fd4d2ab7d24877ca731aa88cea879"}
Oct 30 22:24:26.043364 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:26.043340    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz" event={"ID":"c3682df0-6049-475e-a9b6-2ce93412d5ef","Type":"ContainerDied","Data":"82a40603951e1831f34b12cf9a6133df97b22385f99af4955b0f35b154a09f90"}
Oct 30 22:24:26.043469 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:26.043383    5503 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="82a40603951e1831f34b12cf9a6133df97b22385f99af4955b0f35b154a09f90"
Oct 30 22:24:26.043469 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:26.043347    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85kk5cz"
Oct 30 22:24:29.058233 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:29.058192    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-n8hnc" event={"ID":"7c30467d-1b2e-4ffe-93c9-6eae9212bcee","Type":"ContainerStarted","Data":"17a5c956c9a31792faaf3b880442020f82368dc670edde9e664cd5a63540af96"}
Oct 30 22:24:29.076877 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:29.076824    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-n8hnc" podStartSLOduration=2.5071330659999997 podStartE2EDuration="5.076808953s" totalImagesPullingTime="2.569675887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:29.076428795 +0000 UTC m=+311.345510318" watchObservedRunningTime="2026-10-30 22:24:29.076808953 +0000 UTC m=+311.345890476"
Oct 30 22:24:30.891887 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:30.891845    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:30.998719 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:30.998645    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:43.006987 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:43.006945    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:43.175878 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:43.175830    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:45.389237 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:24:45.389197    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:24:49.051916 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:49.051880    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:49.154748 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:49.154715    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:53.364777 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:53.364725    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:57.620150 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:57.620114    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:57.807714 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:57.807684    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:58.880790 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.880748    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"]
Oct 30 22:24:58.881183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.881117    5503 cpu_manager.go:414] "containerMap: removing container" podUID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerName="util"
Oct 30 22:24:58.881183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.881129    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerName="util"
Oct 30 22:24:58.881183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.881144    5503 cpu_manager.go:414] "containerMap: removing container" podUID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerName="pull"
Oct 30 22:24:58.881183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.881149    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerName="pull"
Oct 30 22:24:58.881183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.881155    5503 cpu_manager.go:414] "containerMap: removing container" podUID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerName="extract"
Oct 30 22:24:58.881183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.881160    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerName="extract"
Oct 30 22:24:58.881357 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.881210    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="c3682df0-6049-475e-a9b6-2ce93412d5ef" containerName="extract"
Oct 30 22:24:58.882939 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.882923    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:58.888915 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.888887    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 22:24:58.888915 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.888900    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 22:24:58.889116 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.888979    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 22:24:58.890385 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.890066    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mwpp7\""
Oct 30 22:24:58.890385 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.890125    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 22:24:58.890385 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.890166    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 22:24:58.890385 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.890205    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 22:24:58.890721 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.890695    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:58.897739 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.897712    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"]
Oct 30 22:24:58.985463 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.985429    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cc6b9-79e0-49d3-8403-65e42254365e-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:58.985657 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.985549    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:58.985657 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.985593    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22f7w\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-kube-api-access-22f7w\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:58.985657 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.985626    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:58.985826 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.985724    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:58.985826 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.985756    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:58.985940 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:58.985854    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.011922 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.011886    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:24:59.087042 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.086999    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.087232 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.087049    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22f7w\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-kube-api-access-22f7w\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.087232 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.087081    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.087232 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.087109    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.087232 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.087140    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.087435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.087236    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.087435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.087261    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cc6b9-79e0-49d3-8403-65e42254365e-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.087920 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.087895    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.089738 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.089717    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.089834 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.089718    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cc6b9-79e0-49d3-8403-65e42254365e-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.090166 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.090146    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.090208 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.090165    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.100323 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.100298    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.100527 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.100505    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22f7w\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-kube-api-access-22f7w\") pod \"istiod-openshift-gateway-7876bfcb74-ldwtr\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.194623 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.194542    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:24:59.478647 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.478564    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"]
Oct 30 22:24:59.481934 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:24:59.481900    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e3cc6b9_79e0_49d3_8403_65e42254365e.slice/crio-2fd42b50895aa370effea9237309260ad7c2bc1aa9de0f7b1a08ef71edd46e41 WatchSource:0}: Error finding container 2fd42b50895aa370effea9237309260ad7c2bc1aa9de0f7b1a08ef71edd46e41: Status 404 returned error can't find the container with id 2fd42b50895aa370effea9237309260ad7c2bc1aa9de0f7b1a08ef71edd46e41
Oct 30 22:24:59.483822 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:24:59.483806    5503 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:25:00.188395 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:00.188356    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr" event={"ID":"5e3cc6b9-79e0-49d3-8403-65e42254365e","Type":"ContainerStarted","Data":"2fd42b50895aa370effea9237309260ad7c2bc1aa9de0f7b1a08ef71edd46e41"}
Oct 30 22:25:00.351920 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:00.351881    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:02.280346 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:02.280311    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:25:02.280771 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:02.280378    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:25:03.014471 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:03.014401    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:03.203603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:03.203569    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr" event={"ID":"5e3cc6b9-79e0-49d3-8403-65e42254365e","Type":"ContainerStarted","Data":"856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c"}
Oct 30 22:25:03.203799 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:03.203777    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:25:03.205702 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:03.205657    5503 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-ldwtr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:25:03.205810 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:03.205763    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr" podUID="5e3cc6b9-79e0-49d3-8403-65e42254365e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:25:03.241896 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:03.241832    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr" podStartSLOduration=2.446477904 podStartE2EDuration="5.241818724s" totalImagesPullingTime="2.79534082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:03.240119682 +0000 UTC m=+345.509201208" watchObservedRunningTime="2026-10-30 22:25:03.241818724 +0000 UTC m=+345.510900246"
Oct 30 22:25:03.250088 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:03.250062    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:04.208920 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:04.208894    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:25:04.665717 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:04.665682    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:05.780404 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.780371    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"]
Oct 30 22:25:05.783956 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.783937    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.798683 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.798643    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"]
Oct 30 22:25:05.837435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.837403    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:05.851615 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.851581    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.851804 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.851627    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7vvz\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-kube-api-access-n7vvz\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.851804 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.851702    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.851804 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.851758    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.851961 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.851827    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.851961 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.851887    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.851961 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.851956    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.852061 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.851979    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.852061 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.852007    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.884391 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.884355    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-kxprg\""
Oct 30 22:25:05.938642 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.938602    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:05.952907 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.952875    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953051 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.952919    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7vvz\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-kube-api-access-n7vvz\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953051 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953035    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953135 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953063    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953135 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953098    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953232 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953163    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953295 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953244    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953295 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953270    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953392 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953327    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953392 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953346    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953494 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953477    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.953760 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953736    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.954015 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.953744    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.954087 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.954066    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.955834 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.955805    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.955926 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.955840    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.969685 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.969644    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:05.969814 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:05.969788    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7vvz\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-kube-api-access-n7vvz\") pod \"openshift-ai-inference-openshift-default-86b669b864-4wfkx\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:06.101056 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:06.101012    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:06.392322 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:06.392295    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"]
Oct 30 22:25:06.394532 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:25:06.394503    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2fe0161_5b10_4d97_97f4_af020ffc5d78.slice/crio-b65d830782be35c248918d59a2da13bad4d7df2b555423ce29f285649668b3df WatchSource:0}: Error finding container b65d830782be35c248918d59a2da13bad4d7df2b555423ce29f285649668b3df: Status 404 returned error can't find the container with id b65d830782be35c248918d59a2da13bad4d7df2b555423ce29f285649668b3df
Oct 30 22:25:07.222204 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:07.222165    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" event={"ID":"c2fe0161-5b10-4d97-97f4-af020ffc5d78","Type":"ContainerStarted","Data":"b65d830782be35c248918d59a2da13bad4d7df2b555423ce29f285649668b3df"}
Oct 30 22:25:08.400278 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:08.400247    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:09.447679 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:09.447633    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:25:09.448017 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:09.447717    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:25:09.448017 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:09.447749    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:25:10.237731 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:10.237687    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" event={"ID":"c2fe0161-5b10-4d97-97f4-af020ffc5d78","Type":"ContainerStarted","Data":"e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f"}
Oct 30 22:25:11.101711 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:11.101656    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:11.106973 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:11.106946    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:11.134524 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:11.134474    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" podStartSLOduration=3.084126562 podStartE2EDuration="6.134440225s" totalImagesPullingTime="3.050313663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:10.26521184 +0000 UTC m=+352.534293433" watchObservedRunningTime="2026-10-30 22:25:11.134440225 +0000 UTC m=+353.403521819"
Oct 30 22:25:11.241734 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:11.241701    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:11.242749 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:11.242731    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:25:11.626590 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:11.626552    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:11.727500 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:11.727465    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:12.534800 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:12.534770    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:14.113654 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.113614    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:14.214869 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.214838    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:14.292429 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.292397    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"]
Oct 30 22:25:14.296228 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.296203    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.302629 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.302604    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 22:25:14.302802 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.302650    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 22:25:14.309718 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.309691    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"]
Oct 30 22:25:14.320265 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.320222    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-b7lxp\""
Oct 30 22:25:14.388256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.388177    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"]
Oct 30 22:25:14.391764 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.391747    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.404250 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.404220    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-v5pd4\""
Oct 30 22:25:14.407148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.407123    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"]
Oct 30 22:25:14.430561 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.430529    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.430757 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.430594    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.430849 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.430761    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr6wq\" (UniqueName: \"kubernetes.io/projected/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-kube-api-access-fr6wq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.508835 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.508794    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"]
Oct 30 22:25:14.512476 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.512459    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.516912 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.516884    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-j247g\""
Oct 30 22:25:14.522680 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.522634    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"]
Oct 30 22:25:14.531734 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.531702    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.531875 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.531773    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.531875 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.531805    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr6wq\" (UniqueName: \"kubernetes.io/projected/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-kube-api-access-fr6wq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.531875 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.531825    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.532029 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.531880    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxglr\" (UniqueName: \"kubernetes.io/projected/347820bd-5078-4582-b714-fb1578b773cc-kube-api-access-qxglr\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.532029 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.531903    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.532135 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.532117    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.532259 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.532241    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.552171 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.552138    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr6wq\" (UniqueName: \"kubernetes.io/projected/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-kube-api-access-fr6wq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.601701 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.601649    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"]
Oct 30 22:25:14.605439 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.605419    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.616064 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.616041    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:14.619881 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.619857    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-xvdc9\""
Oct 30 22:25:14.621316 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.621294    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"]
Oct 30 22:25:14.633356 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.633327    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.633487 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.633366    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.633487 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.633398    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsm4n\" (UniqueName: \"kubernetes.io/projected/90308634-06f9-4943-9cc4-3e07ade5e8fa-kube-api-access-bsm4n\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.633487 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.633432    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.633650 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.633541    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxglr\" (UniqueName: \"kubernetes.io/projected/347820bd-5078-4582-b714-fb1578b773cc-kube-api-access-qxglr\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.633650 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.633626    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.633802 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.633725    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.633957 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.633937    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.650741 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.650650    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxglr\" (UniqueName: \"kubernetes.io/projected/347820bd-5078-4582-b714-fb1578b773cc-kube-api-access-qxglr\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.711025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.710984    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:14.734693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.734639    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.734865 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.734738    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.734865 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.734770    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.734984 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.734957    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srfdn\" (UniqueName: \"kubernetes.io/projected/308d39fc-9e1e-4a48-89f6-84eaca0bc581-kube-api-access-srfdn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.735041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.735006    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.735041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.735027    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsm4n\" (UniqueName: \"kubernetes.io/projected/90308634-06f9-4943-9cc4-3e07ade5e8fa-kube-api-access-bsm4n\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.735147 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.735087    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.735282 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.735266    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.750322 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.750289    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsm4n\" (UniqueName: \"kubernetes.io/projected/90308634-06f9-4943-9cc4-3e07ade5e8fa-kube-api-access-bsm4n\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.831433 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.831398    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:14.835574 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.835395    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.835574 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.835432    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.835574 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.835487    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srfdn\" (UniqueName: \"kubernetes.io/projected/308d39fc-9e1e-4a48-89f6-84eaca0bc581-kube-api-access-srfdn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.835818 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.835792    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.835878 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.835823    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.850830 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.850806    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srfdn\" (UniqueName: \"kubernetes.io/projected/308d39fc-9e1e-4a48-89f6-84eaca0bc581-kube-api-access-srfdn\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.900910 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.900836    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"]
Oct 30 22:25:14.904121 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:25:14.904091    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55fe60fb_dcd5_4c2f_aa64_6f4076643c90.slice/crio-61b72d4b2951a6b2673fa0921dcb9d9090d752e13251349e086c7a7c8aa1bf9d WatchSource:0}: Error finding container 61b72d4b2951a6b2673fa0921dcb9d9090d752e13251349e086c7a7c8aa1bf9d: Status 404 returned error can't find the container with id 61b72d4b2951a6b2673fa0921dcb9d9090d752e13251349e086c7a7c8aa1bf9d
Oct 30 22:25:14.923904 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.923878    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:14.988814 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:14.988777    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"]
Oct 30 22:25:14.993824 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:25:14.993797    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347820bd_5078_4582_b714_fb1578b773cc.slice/crio-74d425e88164cb8ab5c9cfef81df356d9adbfbc79ae68a6b8001a7fb6871b6a9 WatchSource:0}: Error finding container 74d425e88164cb8ab5c9cfef81df356d9adbfbc79ae68a6b8001a7fb6871b6a9: Status 404 returned error can't find the container with id 74d425e88164cb8ab5c9cfef81df356d9adbfbc79ae68a6b8001a7fb6871b6a9
Oct 30 22:25:15.110041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.109885    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"]
Oct 30 22:25:15.112639 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:25:15.112609    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90308634_06f9_4943_9cc4_3e07ade5e8fa.slice/crio-ef38b0b14313d4d5f9dd656f3929bb8ee0d2d3a13d9416630186a66fc38ee1b6 WatchSource:0}: Error finding container ef38b0b14313d4d5f9dd656f3929bb8ee0d2d3a13d9416630186a66fc38ee1b6: Status 404 returned error can't find the container with id ef38b0b14313d4d5f9dd656f3929bb8ee0d2d3a13d9416630186a66fc38ee1b6
Oct 30 22:25:15.209941 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.209911    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"]
Oct 30 22:25:15.227138 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:25:15.227109    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308d39fc_9e1e_4a48_89f6_84eaca0bc581.slice/crio-0b7f4d49552e78e1b15a1f9042b6a3bfef820880951db29a6583289b491b49f8 WatchSource:0}: Error finding container 0b7f4d49552e78e1b15a1f9042b6a3bfef820880951db29a6583289b491b49f8: Status 404 returned error can't find the container with id 0b7f4d49552e78e1b15a1f9042b6a3bfef820880951db29a6583289b491b49f8
Oct 30 22:25:15.263539 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.263505    5503 generic.go:356] "Generic (PLEG): container finished" podID="347820bd-5078-4582-b714-fb1578b773cc" containerID="811d65661723866f9405b06aa656bc41607e5a9fe20e3630d64c476bc4700d30" exitCode=0
Oct 30 22:25:15.263693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.263586    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n" event={"ID":"347820bd-5078-4582-b714-fb1578b773cc","Type":"ContainerDied","Data":"811d65661723866f9405b06aa656bc41607e5a9fe20e3630d64c476bc4700d30"}
Oct 30 22:25:15.263693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.263642    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n" event={"ID":"347820bd-5078-4582-b714-fb1578b773cc","Type":"ContainerStarted","Data":"74d425e88164cb8ab5c9cfef81df356d9adbfbc79ae68a6b8001a7fb6871b6a9"}
Oct 30 22:25:15.265405 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.264928    5503 generic.go:356] "Generic (PLEG): container finished" podID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerID="7754914ae57e23eae088ea071b3a36149714ac2b56e4c900069b046f44893de3" exitCode=0
Oct 30 22:25:15.265405 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.265021    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6" event={"ID":"55fe60fb-dcd5-4c2f-aa64-6f4076643c90","Type":"ContainerDied","Data":"7754914ae57e23eae088ea071b3a36149714ac2b56e4c900069b046f44893de3"}
Oct 30 22:25:15.265405 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.265074    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6" event={"ID":"55fe60fb-dcd5-4c2f-aa64-6f4076643c90","Type":"ContainerStarted","Data":"61b72d4b2951a6b2673fa0921dcb9d9090d752e13251349e086c7a7c8aa1bf9d"}
Oct 30 22:25:15.267700 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.266871    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr" event={"ID":"308d39fc-9e1e-4a48-89f6-84eaca0bc581","Type":"ContainerStarted","Data":"0b7f4d49552e78e1b15a1f9042b6a3bfef820880951db29a6583289b491b49f8"}
Oct 30 22:25:15.268856 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.268825    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz" event={"ID":"90308634-06f9-4943-9cc4-3e07ade5e8fa","Type":"ContainerStarted","Data":"36338397c0dbe5b268442cce05441ef35c36cd21058f4ab72b64f6deeee1f59f"}
Oct 30 22:25:15.268941 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:15.268884    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz" event={"ID":"90308634-06f9-4943-9cc4-3e07ade5e8fa","Type":"ContainerStarted","Data":"ef38b0b14313d4d5f9dd656f3929bb8ee0d2d3a13d9416630186a66fc38ee1b6"}
Oct 30 22:25:15.389321 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:25:15.389287    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:25:16.275183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:16.275150    5503 generic.go:356] "Generic (PLEG): container finished" podID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerID="a3cd0cb9c010a526af84c946698222475edc054673c3f03034c82b3fd53a5b8c" exitCode=0
Oct 30 22:25:16.275606 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:16.275240    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr" event={"ID":"308d39fc-9e1e-4a48-89f6-84eaca0bc581","Type":"ContainerDied","Data":"a3cd0cb9c010a526af84c946698222475edc054673c3f03034c82b3fd53a5b8c"}
Oct 30 22:25:16.276869 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:16.276839    5503 generic.go:356] "Generic (PLEG): container finished" podID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerID="36338397c0dbe5b268442cce05441ef35c36cd21058f4ab72b64f6deeee1f59f" exitCode=0
Oct 30 22:25:16.276998 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:16.276945    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz" event={"ID":"90308634-06f9-4943-9cc4-3e07ade5e8fa","Type":"ContainerDied","Data":"36338397c0dbe5b268442cce05441ef35c36cd21058f4ab72b64f6deeee1f59f"}
Oct 30 22:25:17.289408 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:17.289372    5503 generic.go:356] "Generic (PLEG): container finished" podID="347820bd-5078-4582-b714-fb1578b773cc" containerID="92ef02583cebe9ba86a3b390ec0f2bb5d751c073eb1c7c1401c1d910b855d61f" exitCode=0
Oct 30 22:25:17.289904 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:17.289442    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n" event={"ID":"347820bd-5078-4582-b714-fb1578b773cc","Type":"ContainerDied","Data":"92ef02583cebe9ba86a3b390ec0f2bb5d751c073eb1c7c1401c1d910b855d61f"}
Oct 30 22:25:17.291461 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:17.291429    5503 generic.go:356] "Generic (PLEG): container finished" podID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerID="a2e02de527454206d8927309b1e9271a3348868c55978f8a77e8cb165557090e" exitCode=0
Oct 30 22:25:17.291569 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:17.291536    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6" event={"ID":"55fe60fb-dcd5-4c2f-aa64-6f4076643c90","Type":"ContainerDied","Data":"a2e02de527454206d8927309b1e9271a3348868c55978f8a77e8cb165557090e"}
Oct 30 22:25:18.298333 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:18.298301    5503 generic.go:356] "Generic (PLEG): container finished" podID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerID="196c4155fcf2360647fbb393ea57463e0c1221f2b20b6132ed3fc7ebc9a1d836" exitCode=0
Oct 30 22:25:18.298789 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:18.298383    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr" event={"ID":"308d39fc-9e1e-4a48-89f6-84eaca0bc581","Type":"ContainerDied","Data":"196c4155fcf2360647fbb393ea57463e0c1221f2b20b6132ed3fc7ebc9a1d836"}
Oct 30 22:25:18.300310 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:18.300287    5503 generic.go:356] "Generic (PLEG): container finished" podID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerID="6c99e72bc551b253c09d587c620ceee48cd55aafc096326527fd9fe5de1f0d5b" exitCode=0
Oct 30 22:25:18.300394 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:18.300319    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz" event={"ID":"90308634-06f9-4943-9cc4-3e07ade5e8fa","Type":"ContainerDied","Data":"6c99e72bc551b253c09d587c620ceee48cd55aafc096326527fd9fe5de1f0d5b"}
Oct 30 22:25:18.303087 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:18.303067    5503 generic.go:356] "Generic (PLEG): container finished" podID="347820bd-5078-4582-b714-fb1578b773cc" containerID="920c8c6fa2bbaa19f66ce3673de7bbfdf230a944a997c6152bc2f6546dfb90a7" exitCode=0
Oct 30 22:25:18.303185 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:18.303148    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n" event={"ID":"347820bd-5078-4582-b714-fb1578b773cc","Type":"ContainerDied","Data":"920c8c6fa2bbaa19f66ce3673de7bbfdf230a944a997c6152bc2f6546dfb90a7"}
Oct 30 22:25:18.305315 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:18.305287    5503 generic.go:356] "Generic (PLEG): container finished" podID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerID="52235b373015a0590554248bdb581658c351b798145227ec4f707cb502c0ac5c" exitCode=0
Oct 30 22:25:18.305417 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:18.305317    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6" event={"ID":"55fe60fb-dcd5-4c2f-aa64-6f4076643c90","Type":"ContainerDied","Data":"52235b373015a0590554248bdb581658c351b798145227ec4f707cb502c0ac5c"}
Oct 30 22:25:19.312808 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.312771    5503 generic.go:356] "Generic (PLEG): container finished" podID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerID="2e8832dae410458270f23f87e3f87cacccd41994898011cd788930cb8005593f" exitCode=0
Oct 30 22:25:19.313258 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.312857    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr" event={"ID":"308d39fc-9e1e-4a48-89f6-84eaca0bc581","Type":"ContainerDied","Data":"2e8832dae410458270f23f87e3f87cacccd41994898011cd788930cb8005593f"}
Oct 30 22:25:19.314872 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.314848    5503 generic.go:356] "Generic (PLEG): container finished" podID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerID="1c032592eb27dea258f1c970918d6b51a3eb3e8fb578de2204a8116342f88723" exitCode=0
Oct 30 22:25:19.314953 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.314931    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz" event={"ID":"90308634-06f9-4943-9cc4-3e07ade5e8fa","Type":"ContainerDied","Data":"1c032592eb27dea258f1c970918d6b51a3eb3e8fb578de2204a8116342f88723"}
Oct 30 22:25:19.456002 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.455970    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:19.477442 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.477169    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:19.575473 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.575386    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-kube-api-access-fr6wq\" (UniqueName: \"kubernetes.io/projected/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-kube-api-access-fr6wq\") pod \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") "
Oct 30 22:25:19.575473 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.575438    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-bundle\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-bundle\") pod \"347820bd-5078-4582-b714-fb1578b773cc\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") "
Oct 30 22:25:19.575746 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.575491    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/347820bd-5078-4582-b714-fb1578b773cc-kube-api-access-qxglr\" (UniqueName: \"kubernetes.io/projected/347820bd-5078-4582-b714-fb1578b773cc-kube-api-access-qxglr\") pod \"347820bd-5078-4582-b714-fb1578b773cc\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") "
Oct 30 22:25:19.575746 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.575526    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-util\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-util\") pod \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") "
Oct 30 22:25:19.575746 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.575554    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-bundle\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-bundle\") pod \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\" (UID: \"55fe60fb-dcd5-4c2f-aa64-6f4076643c90\") "
Oct 30 22:25:19.575746 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.575587    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-util\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-util\") pod \"347820bd-5078-4582-b714-fb1578b773cc\" (UID: \"347820bd-5078-4582-b714-fb1578b773cc\") "
Oct 30 22:25:19.576124 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.576094    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-bundle" pod "347820bd-5078-4582-b714-fb1578b773cc" (UID: "347820bd-5078-4582-b714-fb1578b773cc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:19.576211 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.576143    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-bundle" pod "55fe60fb-dcd5-4c2f-aa64-6f4076643c90" (UID: "55fe60fb-dcd5-4c2f-aa64-6f4076643c90"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:19.577911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.577883    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/347820bd-5078-4582-b714-fb1578b773cc-kube-api-access-qxglr" pod "347820bd-5078-4582-b714-fb1578b773cc" (UID: "347820bd-5078-4582-b714-fb1578b773cc"). InnerVolumeSpecName "kube-api-access-qxglr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:19.578056 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.578039    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-kube-api-access-fr6wq" pod "55fe60fb-dcd5-4c2f-aa64-6f4076643c90" (UID: "55fe60fb-dcd5-4c2f-aa64-6f4076643c90"). InnerVolumeSpecName "kube-api-access-fr6wq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:19.581377 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.581340    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-util" pod "55fe60fb-dcd5-4c2f-aa64-6f4076643c90" (UID: "55fe60fb-dcd5-4c2f-aa64-6f4076643c90"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:19.582124 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.582101    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-util" pod "347820bd-5078-4582-b714-fb1578b773cc" (UID: "347820bd-5078-4582-b714-fb1578b773cc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:19.676608 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.676570    5503 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-bundle\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:19.676608 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.676602    5503 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-util\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:19.676608 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.676612    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fr6wq\" (UniqueName: \"kubernetes.io/projected/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-kube-api-access-fr6wq\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:19.676865 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.676623    5503 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/347820bd-5078-4582-b714-fb1578b773cc-bundle\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:19.676865 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.676632    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxglr\" (UniqueName: \"kubernetes.io/projected/347820bd-5078-4582-b714-fb1578b773cc-kube-api-access-qxglr\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:19.676865 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:19.676641    5503 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55fe60fb-dcd5-4c2f-aa64-6f4076643c90-util\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:20.321700 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.321579    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n" event={"ID":"347820bd-5078-4582-b714-fb1578b773cc","Type":"ContainerDied","Data":"74d425e88164cb8ab5c9cfef81df356d9adbfbc79ae68a6b8001a7fb6871b6a9"}
Oct 30 22:25:20.321700 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.321648    5503 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74d425e88164cb8ab5c9cfef81df356d9adbfbc79ae68a6b8001a7fb6871b6a9"
Oct 30 22:25:20.321700 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.321591    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7fzs9n"
Oct 30 22:25:20.323756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.323723    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6"
Oct 30 22:25:20.323892 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.323771    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c64xws6" event={"ID":"55fe60fb-dcd5-4c2f-aa64-6f4076643c90","Type":"ContainerDied","Data":"61b72d4b2951a6b2673fa0921dcb9d9090d752e13251349e086c7a7c8aa1bf9d"}
Oct 30 22:25:20.323892 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.323833    5503 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61b72d4b2951a6b2673fa0921dcb9d9090d752e13251349e086c7a7c8aa1bf9d"
Oct 30 22:25:20.480999 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.480968    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:20.484091 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.484061    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:20.585206 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.585103    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-util\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-util\") pod \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") "
Oct 30 22:25:20.585206 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.585159    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90308634-06f9-4943-9cc4-3e07ade5e8fa-kube-api-access-bsm4n\" (UniqueName: \"kubernetes.io/projected/90308634-06f9-4943-9cc4-3e07ade5e8fa-kube-api-access-bsm4n\") pod \"90308634-06f9-4943-9cc4-3e07ade5e8fa\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") "
Oct 30 22:25:20.585206 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.585187    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/308d39fc-9e1e-4a48-89f6-84eaca0bc581-kube-api-access-srfdn\" (UniqueName: \"kubernetes.io/projected/308d39fc-9e1e-4a48-89f6-84eaca0bc581-kube-api-access-srfdn\") pod \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") "
Oct 30 22:25:20.585493 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.585262    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-util\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-util\") pod \"90308634-06f9-4943-9cc4-3e07ade5e8fa\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") "
Oct 30 22:25:20.585493 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.585386    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-bundle\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-bundle\") pod \"90308634-06f9-4943-9cc4-3e07ade5e8fa\" (UID: \"90308634-06f9-4943-9cc4-3e07ade5e8fa\") "
Oct 30 22:25:20.585493 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.585476    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-bundle\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-bundle\") pod \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\" (UID: \"308d39fc-9e1e-4a48-89f6-84eaca0bc581\") "
Oct 30 22:25:20.586091 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.586061    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-bundle" pod "308d39fc-9e1e-4a48-89f6-84eaca0bc581" (UID: "308d39fc-9e1e-4a48-89f6-84eaca0bc581"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:20.586178 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.586150    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-bundle" pod "90308634-06f9-4943-9cc4-3e07ade5e8fa" (UID: "90308634-06f9-4943-9cc4-3e07ade5e8fa"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:20.587739 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.587711    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90308634-06f9-4943-9cc4-3e07ade5e8fa-kube-api-access-bsm4n" pod "90308634-06f9-4943-9cc4-3e07ade5e8fa" (UID: "90308634-06f9-4943-9cc4-3e07ade5e8fa"). InnerVolumeSpecName "kube-api-access-bsm4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:20.587945 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.587925    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/308d39fc-9e1e-4a48-89f6-84eaca0bc581-kube-api-access-srfdn" pod "308d39fc-9e1e-4a48-89f6-84eaca0bc581" (UID: "308d39fc-9e1e-4a48-89f6-84eaca0bc581"). InnerVolumeSpecName "kube-api-access-srfdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:25:20.590767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.590722    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-util" pod "90308634-06f9-4943-9cc4-3e07ade5e8fa" (UID: "90308634-06f9-4943-9cc4-3e07ade5e8fa"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:20.590860 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.590842    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-util" pod "308d39fc-9e1e-4a48-89f6-84eaca0bc581" (UID: "308d39fc-9e1e-4a48-89f6-84eaca0bc581"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:25:20.686545 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.686508    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsm4n\" (UniqueName: \"kubernetes.io/projected/90308634-06f9-4943-9cc4-3e07ade5e8fa-kube-api-access-bsm4n\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:20.686545 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.686539    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srfdn\" (UniqueName: \"kubernetes.io/projected/308d39fc-9e1e-4a48-89f6-84eaca0bc581-kube-api-access-srfdn\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:20.686545 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.686548    5503 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-util\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:20.686821 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.686558    5503 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/90308634-06f9-4943-9cc4-3e07ade5e8fa-bundle\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:20.686821 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.686567    5503 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-bundle\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:20.686821 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:20.686574    5503 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/308d39fc-9e1e-4a48-89f6-84eaca0bc581-util\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:25:21.330353 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:21.330314    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr" event={"ID":"308d39fc-9e1e-4a48-89f6-84eaca0bc581","Type":"ContainerDied","Data":"0b7f4d49552e78e1b15a1f9042b6a3bfef820880951db29a6583289b491b49f8"}
Oct 30 22:25:21.330796 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:21.330364    5503 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0b7f4d49552e78e1b15a1f9042b6a3bfef820880951db29a6583289b491b49f8"
Oct 30 22:25:21.330796 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:21.330329    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cxspzr"
Oct 30 22:25:21.332327 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:21.332306    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz"
Oct 30 22:25:21.332327 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:21.332316    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847adlcpz" event={"ID":"90308634-06f9-4943-9cc4-3e07ade5e8fa","Type":"ContainerDied","Data":"ef38b0b14313d4d5f9dd656f3929bb8ee0d2d3a13d9416630186a66fc38ee1b6"}
Oct 30 22:25:21.332465 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:21.332352    5503 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef38b0b14313d4d5f9dd656f3929bb8ee0d2d3a13d9416630186a66fc38ee1b6"
Oct 30 22:25:23.465878 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:23.465847    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:24.253628 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:24.253593    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:26.701057 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701020    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-66498fdf6c-b8722"]
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701350    5503 cpu_manager.go:414] "containerMap: removing container" podUID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerName="util"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701361    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerName="util"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701378    5503 cpu_manager.go:414] "containerMap: removing container" podUID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerName="util"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701383    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerName="util"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701388    5503 cpu_manager.go:414] "containerMap: removing container" podUID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerName="extract"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701393    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerName="extract"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701400    5503 cpu_manager.go:414] "containerMap: removing container" podUID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerName="pull"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701404    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerName="pull"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701410    5503 cpu_manager.go:414] "containerMap: removing container" podUID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerName="pull"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701414    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerName="pull"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701420    5503 cpu_manager.go:414] "containerMap: removing container" podUID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerName="extract"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701423    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerName="extract"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701433    5503 cpu_manager.go:414] "containerMap: removing container" podUID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerName="util"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701437    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerName="util"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701442    5503 cpu_manager.go:414] "containerMap: removing container" podUID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerName="extract"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701445    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerName="extract"
Oct 30 22:25:26.701444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701453    5503 cpu_manager.go:414] "containerMap: removing container" podUID="347820bd-5078-4582-b714-fb1578b773cc" containerName="extract"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701458    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="347820bd-5078-4582-b714-fb1578b773cc" containerName="extract"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701467    5503 cpu_manager.go:414] "containerMap: removing container" podUID="347820bd-5078-4582-b714-fb1578b773cc" containerName="util"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701471    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="347820bd-5078-4582-b714-fb1578b773cc" containerName="util"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701479    5503 cpu_manager.go:414] "containerMap: removing container" podUID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerName="pull"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701485    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerName="pull"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701493    5503 cpu_manager.go:414] "containerMap: removing container" podUID="347820bd-5078-4582-b714-fb1578b773cc" containerName="pull"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701497    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="347820bd-5078-4582-b714-fb1578b773cc" containerName="pull"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701546    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="308d39fc-9e1e-4a48-89f6-84eaca0bc581" containerName="extract"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701553    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="55fe60fb-dcd5-4c2f-aa64-6f4076643c90" containerName="extract"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701560    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="90308634-06f9-4943-9cc4-3e07ade5e8fa" containerName="extract"
Oct 30 22:25:26.701931 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.701565    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="347820bd-5078-4582-b714-fb1578b773cc" containerName="extract"
Oct 30 22:25:26.706338 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.706320    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-66498fdf6c-b8722"
Oct 30 22:25:26.717120 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.717091    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 22:25:26.717337 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.717284    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 22:25:26.718961 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.718940    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-66498fdf6c-b8722"]
Oct 30 22:25:26.787395 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.787363    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-p7nvs\""
Oct 30 22:25:26.840946 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.840903    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96pp9\" (UniqueName: \"kubernetes.io/projected/66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e-kube-api-access-96pp9\") pod \"authorino-operator-66498fdf6c-b8722\" (UID: \"66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e\") " pod="kuadrant-system/authorino-operator-66498fdf6c-b8722"
Oct 30 22:25:26.941795 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.941707    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pp9\" (UniqueName: \"kubernetes.io/projected/66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e-kube-api-access-96pp9\") pod \"authorino-operator-66498fdf6c-b8722\" (UID: \"66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e\") " pod="kuadrant-system/authorino-operator-66498fdf6c-b8722"
Oct 30 22:25:26.955454 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:26.955372    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96pp9\" (UniqueName: \"kubernetes.io/projected/66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e-kube-api-access-96pp9\") pod \"authorino-operator-66498fdf6c-b8722\" (UID: \"66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e\") " pod="kuadrant-system/authorino-operator-66498fdf6c-b8722"
Oct 30 22:25:27.019205 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:27.019171    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-66498fdf6c-b8722"
Oct 30 22:25:27.300105 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:27.300079    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-66498fdf6c-b8722"]
Oct 30 22:25:27.302512 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:25:27.302486    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66d6e6c1_1a6a_4b1f_b7c8_04aabd0bf00e.slice/crio-a3e5d205a43823018d707c268cc53200201c1e25934b06a5e6756b526b41a625 WatchSource:0}: Error finding container a3e5d205a43823018d707c268cc53200201c1e25934b06a5e6756b526b41a625: Status 404 returned error can't find the container with id a3e5d205a43823018d707c268cc53200201c1e25934b06a5e6756b526b41a625
Oct 30 22:25:27.333059 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:27.333026    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:27.363829 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:27.363791    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-66498fdf6c-b8722" event={"ID":"66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e","Type":"ContainerStarted","Data":"a3e5d205a43823018d707c268cc53200201c1e25934b06a5e6756b526b41a625"}
Oct 30 22:25:27.454351 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:27.454308    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:27.737961 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:27.737901    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:27.905761 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:27.905720    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:28.935411 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:28.935372    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:29.105195 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:29.105158    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:30.235475 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:30.235444    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:30.379551 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:30.379508    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-66498fdf6c-b8722" event={"ID":"66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e","Type":"ContainerStarted","Data":"207506bfad827f1b731150fdfebfb37ca7cc07cbb261cffa1855b4717d8f6467"}
Oct 30 22:25:30.379731 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:30.379582    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-66498fdf6c-b8722"
Oct 30 22:25:30.403588 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:30.403525    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-66498fdf6c-b8722" podStartSLOduration=1.439195884 podStartE2EDuration="4.403505076s" totalImagesPullingTime="2.964309192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:30.403020153 +0000 UTC m=+372.672101694" watchObservedRunningTime="2026-10-30 22:25:30.403505076 +0000 UTC m=+372.672586600"
Oct 30 22:25:31.069438 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:31.069410    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:33.067084 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:33.067053    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:33.482828 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:33.482783    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:33.837692 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:33.837598    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:34.376409 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:34.376383    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:34.494178 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:34.494147    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:38.436754 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:38.436659    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:38.540736 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:38.540693    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:38.962169 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:38.962132    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:39.064630 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:39.064599    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:39.164772 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:39.164729    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:41.387615 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:41.387579    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-66498fdf6c-b8722"
Oct 30 22:25:41.465486 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:41.465453    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:41.566235 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:41.566204    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:44.454160 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:44.454127    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:44.555728 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:44.555693    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:45.389447 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:25:45.389413    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:25:45.714346 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:45.714265    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:45.830767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:45.830735    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:49.105763 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:49.105729    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:50.473007 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:50.472947    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:50.571734 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:50.571703    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:52.944252 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:52.944220    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:53.043250 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:53.043218    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:57.502683 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:57.502630    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:57.596799 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:57.596761    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:57.923042 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:57.923010    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:58.024504 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:58.024468    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:25:59.028029 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:25:59.027995    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:01.122387 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:01.122336    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:01.224012 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:01.223981    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:05.128165 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:05.128130    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:05.233181 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:05.233147    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:09.939125 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:09.939091    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:10.041494 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:10.041455    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:13.541110 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:13.541078    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:13.651721 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:13.651658    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:15.389499 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:26:15.389425    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:26:40.258085 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.258044    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"]
Oct 30 22:26:40.261346 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.261326    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:40.271766 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.271740    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 22:26:40.273116 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.273092    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"]
Oct 30 22:26:40.273781 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.273763    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-b87l4\""
Oct 30 22:26:40.284622 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.284590    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:40.359543 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.359511    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"]
Oct 30 22:26:40.385053 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.385024    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:40.395624 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.395583    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-config-file\") pod \"limitador-limitador-74dfcfb894-5bvzn\" (UID: \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:40.395624 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.395626    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrs7r\" (UniqueName: \"kubernetes.io/projected/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-kube-api-access-nrs7r\") pod \"limitador-limitador-74dfcfb894-5bvzn\" (UID: \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:40.484777 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.484741    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:40.496629 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.496600    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-config-file\") pod \"limitador-limitador-74dfcfb894-5bvzn\" (UID: \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:40.496751 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.496644    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrs7r\" (UniqueName: \"kubernetes.io/projected/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-kube-api-access-nrs7r\") pod \"limitador-limitador-74dfcfb894-5bvzn\" (UID: \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:40.497333 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.497313    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-config-file\") pod \"limitador-limitador-74dfcfb894-5bvzn\" (UID: \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:40.513869 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.513796    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrs7r\" (UniqueName: \"kubernetes.io/projected/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-kube-api-access-nrs7r\") pod \"limitador-limitador-74dfcfb894-5bvzn\" (UID: \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:40.574320 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.574285    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:40.850767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:40.850738    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"]
Oct 30 22:26:40.853525 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:26:40.853496    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa2bf20f_7b86_446c_8d00_cc5ee9a10fc3.slice/crio-23f621094e1bb0b004857a1d7ab8dfc51f5d1ef7102cc46a511a2d0104f2f556 WatchSource:0}: Error finding container 23f621094e1bb0b004857a1d7ab8dfc51f5d1ef7102cc46a511a2d0104f2f556: Status 404 returned error can't find the container with id 23f621094e1bb0b004857a1d7ab8dfc51f5d1ef7102cc46a511a2d0104f2f556
Oct 30 22:26:41.171164 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:41.171083    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:41.303958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:41.303922    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:41.722641 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:41.722608    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn" event={"ID":"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3","Type":"ContainerStarted","Data":"23f621094e1bb0b004857a1d7ab8dfc51f5d1ef7102cc46a511a2d0104f2f556"}
Oct 30 22:26:42.728429 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:42.728344    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn" event={"ID":"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3","Type":"ContainerStarted","Data":"44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d"}
Oct 30 22:26:42.728825 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:42.728467    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:42.748422 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:42.748375    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn" podStartSLOduration=1.233404381 podStartE2EDuration="2.7483603s" totalImagesPullingTime="1.514955919s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:26:42.747901464 +0000 UTC m=+445.016982989" watchObservedRunningTime="2026-10-30 22:26:42.7483603 +0000 UTC m=+445.017441823"
Oct 30 22:26:45.389508 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:26:45.389468    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:26:49.162184 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:49.162147    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:26:53.734716 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:53.734657    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:54.515384 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:54.515351    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"]
Oct 30 22:26:54.515584 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:54.515560    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn" podUID="aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3" containerName="limitador" containerID="cri-o://44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d" gracePeriod=30
Oct 30 22:26:55.058426 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.058391    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:55.237797 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.237760    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-config-file\" (UniqueName: \"kubernetes.io/configmap/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-config-file\") pod \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\" (UID: \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\") "
Oct 30 22:26:55.237999 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.237883    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-kube-api-access-nrs7r\" (UniqueName: \"kubernetes.io/projected/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-kube-api-access-nrs7r\") pod \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\" (UID: \"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3\") "
Oct 30 22:26:55.238215 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.238191    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-config-file" pod "aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3" (UID: "aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:26:55.240246 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.240220    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-kube-api-access-nrs7r" pod "aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3" (UID: "aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3"). InnerVolumeSpecName "kube-api-access-nrs7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:26:55.338860 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.338818    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrs7r\" (UniqueName: \"kubernetes.io/projected/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-kube-api-access-nrs7r\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:55.338860 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.338856    5503 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3-config-file\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:26:55.789144 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.789106    5503 generic.go:356] "Generic (PLEG): container finished" podID="aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3" containerID="44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d" exitCode=0
Oct 30 22:26:55.789333 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.789158    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"
Oct 30 22:26:55.789333 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.789185    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn" event={"ID":"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3","Type":"ContainerDied","Data":"44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d"}
Oct 30 22:26:55.789333 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.789266    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-5bvzn" event={"ID":"aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3","Type":"ContainerDied","Data":"23f621094e1bb0b004857a1d7ab8dfc51f5d1ef7102cc46a511a2d0104f2f556"}
Oct 30 22:26:55.789333 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.789294    5503 scope.go:122] "RemoveContainer" containerID="44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d"
Oct 30 22:26:55.798473 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.798451    5503 scope.go:122] "RemoveContainer" containerID="44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d"
Oct 30 22:26:55.798779 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:26:55.798761    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d\": container with ID starting with 44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d not found: ID does not exist" containerID="44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d"
Oct 30 22:26:55.798847 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.798788    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d"} err="failed to get container status \"44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d\": rpc error: code = NotFound desc = could not find container \"44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d\": container with ID starting with 44c3921cec971f56b302789e0b2a31a75f1885c0271d8b8847d3aeae64fbad0d not found: ID does not exist"
Oct 30 22:26:55.812191 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.812162    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"]
Oct 30 22:26:55.816558 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.816532    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-5bvzn"]
Oct 30 22:26:55.854474 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:26:55.854445    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3" path="/var/lib/kubelet/pods/aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3/volumes"
Oct 30 22:27:01.062890 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:01.062853    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:01.374862 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:01.374829    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:05.619480 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.619444    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557c8f8584-6kpnv"]
Oct 30 22:27:05.619896 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.619829    5503 cpu_manager.go:414] "containerMap: removing container" podUID="aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3" containerName="limitador"
Oct 30 22:27:05.619896 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.619840    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3" containerName="limitador"
Oct 30 22:27:05.619964 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.619923    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa2bf20f-7b86-446c-8d00-cc5ee9a10fc3" containerName="limitador"
Oct 30 22:27:05.622874 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.622857    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557c8f8584-6kpnv"
Oct 30 22:27:05.639206 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.639176    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557c8f8584-6kpnv"]
Oct 30 22:27:05.642741 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.642718    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 22:27:05.643713 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.643696    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-rr875\""
Oct 30 22:27:05.729790 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.729745    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b73480bd-92cc-4f2d-a4ed-c794d61edab0-tls-cert\") pod \"authorino-557c8f8584-6kpnv\" (UID: \"b73480bd-92cc-4f2d-a4ed-c794d61edab0\") " pod="kuadrant-system/authorino-557c8f8584-6kpnv"
Oct 30 22:27:05.729963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.729888    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wt9bm\" (UniqueName: \"kubernetes.io/projected/b73480bd-92cc-4f2d-a4ed-c794d61edab0-kube-api-access-wt9bm\") pod \"authorino-557c8f8584-6kpnv\" (UID: \"b73480bd-92cc-4f2d-a4ed-c794d61edab0\") " pod="kuadrant-system/authorino-557c8f8584-6kpnv"
Oct 30 22:27:05.830681 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.830630    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wt9bm\" (UniqueName: \"kubernetes.io/projected/b73480bd-92cc-4f2d-a4ed-c794d61edab0-kube-api-access-wt9bm\") pod \"authorino-557c8f8584-6kpnv\" (UID: \"b73480bd-92cc-4f2d-a4ed-c794d61edab0\") " pod="kuadrant-system/authorino-557c8f8584-6kpnv"
Oct 30 22:27:05.830859 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.830756    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b73480bd-92cc-4f2d-a4ed-c794d61edab0-tls-cert\") pod \"authorino-557c8f8584-6kpnv\" (UID: \"b73480bd-92cc-4f2d-a4ed-c794d61edab0\") " pod="kuadrant-system/authorino-557c8f8584-6kpnv"
Oct 30 22:27:05.833445 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.833416    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b73480bd-92cc-4f2d-a4ed-c794d61edab0-tls-cert\") pod \"authorino-557c8f8584-6kpnv\" (UID: \"b73480bd-92cc-4f2d-a4ed-c794d61edab0\") " pod="kuadrant-system/authorino-557c8f8584-6kpnv"
Oct 30 22:27:05.844151 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.844125    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wt9bm\" (UniqueName: \"kubernetes.io/projected/b73480bd-92cc-4f2d-a4ed-c794d61edab0-kube-api-access-wt9bm\") pod \"authorino-557c8f8584-6kpnv\" (UID: \"b73480bd-92cc-4f2d-a4ed-c794d61edab0\") " pod="kuadrant-system/authorino-557c8f8584-6kpnv"
Oct 30 22:27:05.934795 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:05.934715    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557c8f8584-6kpnv"
Oct 30 22:27:06.209256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:06.209229    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557c8f8584-6kpnv"]
Oct 30 22:27:06.211001 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:27:06.210969    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb73480bd_92cc_4f2d_a4ed_c794d61edab0.slice/crio-6ad6d8159bd47425c4350b870ee504c49cfcad46e3749704bf0da7eba2b52d40 WatchSource:0}: Error finding container 6ad6d8159bd47425c4350b870ee504c49cfcad46e3749704bf0da7eba2b52d40: Status 404 returned error can't find the container with id 6ad6d8159bd47425c4350b870ee504c49cfcad46e3749704bf0da7eba2b52d40
Oct 30 22:27:06.844596 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:06.844559    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557c8f8584-6kpnv" event={"ID":"b73480bd-92cc-4f2d-a4ed-c794d61edab0","Type":"ContainerStarted","Data":"6ad6d8159bd47425c4350b870ee504c49cfcad46e3749704bf0da7eba2b52d40"}
Oct 30 22:27:08.856024 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:08.855986    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557c8f8584-6kpnv" event={"ID":"b73480bd-92cc-4f2d-a4ed-c794d61edab0","Type":"ContainerStarted","Data":"361d285a0657e38028291360fcc8d409711d7c0ddf59d98ab9e4670220321b00"}
Oct 30 22:27:08.874349 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:08.874295    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-557c8f8584-6kpnv" podStartSLOduration=1.58747444 podStartE2EDuration="3.874278971s" totalImagesPullingTime="2.286804531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:27:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:27:08.873176735 +0000 UTC m=+471.142258259" watchObservedRunningTime="2026-10-30 22:27:08.874278971 +0000 UTC m=+471.143360494"
Oct 30 22:27:09.077687 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:09.077636    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:11.749334 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:11.749297    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:11.849678 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:11.849644    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:15.389315 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:27:15.389276    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:27:28.477389 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.477349    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"]
Oct 30 22:27:28.480730 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.480713    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.498168 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.498130    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"]
Oct 30 22:27:28.524857 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.524821    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/40a5acae-35b7-4ac0-911b-7699d0127186-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.525022 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.524861    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkk44\" (UniqueName: \"kubernetes.io/projected/40a5acae-35b7-4ac0-911b-7699d0127186-kube-api-access-dkk44\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.525022 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.524932    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/40a5acae-35b7-4ac0-911b-7699d0127186-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.525022 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.525007    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/40a5acae-35b7-4ac0-911b-7699d0127186-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.525143 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.525041    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40a5acae-35b7-4ac0-911b-7699d0127186-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.525143 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.525057    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40a5acae-35b7-4ac0-911b-7699d0127186-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.525143 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.525126    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/40a5acae-35b7-4ac0-911b-7699d0127186-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.625553 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.625517    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/40a5acae-35b7-4ac0-911b-7699d0127186-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.625553 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.625557    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkk44\" (UniqueName: \"kubernetes.io/projected/40a5acae-35b7-4ac0-911b-7699d0127186-kube-api-access-dkk44\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.625779 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.625706    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/40a5acae-35b7-4ac0-911b-7699d0127186-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.625779 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.625752    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/40a5acae-35b7-4ac0-911b-7699d0127186-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.625779 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.625772    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40a5acae-35b7-4ac0-911b-7699d0127186-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.625909 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.625803    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40a5acae-35b7-4ac0-911b-7699d0127186-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.625909 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.625834    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/40a5acae-35b7-4ac0-911b-7699d0127186-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.626790 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.626744    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/40a5acae-35b7-4ac0-911b-7699d0127186-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.627990 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.627962    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/40a5acae-35b7-4ac0-911b-7699d0127186-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.628252 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.628231    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/40a5acae-35b7-4ac0-911b-7699d0127186-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.628347 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.628307    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/40a5acae-35b7-4ac0-911b-7699d0127186-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.628524 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.628504    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/40a5acae-35b7-4ac0-911b-7699d0127186-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.638729 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.638701    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40a5acae-35b7-4ac0-911b-7699d0127186-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.638906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.638889    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkk44\" (UniqueName: \"kubernetes.io/projected/40a5acae-35b7-4ac0-911b-7699d0127186-kube-api-access-dkk44\") pod \"istiod-openshift-gateway-7ddd9cfdc8-wr97s\" (UID: \"40a5acae-35b7-4ac0-911b-7699d0127186\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:28.792219 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:28.792125    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:29.076784 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:29.076713    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"]
Oct 30 22:27:29.080034 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:27:29.080002    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a5acae_35b7_4ac0_911b_7699d0127186.slice/crio-aa9e8ce67962235b69f4693b70e612a509ac8c6a7c12338e096397a3a93be31f WatchSource:0}: Error finding container aa9e8ce67962235b69f4693b70e612a509ac8c6a7c12338e096397a3a93be31f: Status 404 returned error can't find the container with id aa9e8ce67962235b69f4693b70e612a509ac8c6a7c12338e096397a3a93be31f
Oct 30 22:27:29.082091 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:29.082062    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:27:29.082153 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:29.082133    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:27:29.952695 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:29.952634    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s" event={"ID":"40a5acae-35b7-4ac0-911b-7699d0127186","Type":"ContainerStarted","Data":"1b058a317fbc4b95e8db825a4f9abd2348bdb2d735f2f0f20456a92e68654b18"}
Oct 30 22:27:29.953193 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:29.952715    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s" event={"ID":"40a5acae-35b7-4ac0-911b-7699d0127186","Type":"ContainerStarted","Data":"aa9e8ce67962235b69f4693b70e612a509ac8c6a7c12338e096397a3a93be31f"}
Oct 30 22:27:29.953193 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:29.952794    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:29.979838 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:29.979788    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s" podStartSLOduration=1.9797711009999999 podStartE2EDuration="1.979771101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:27:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:27:29.977131064 +0000 UTC m=+492.246212600" watchObservedRunningTime="2026-10-30 22:27:29.979771101 +0000 UTC m=+492.248852623"
Oct 30 22:27:30.959340 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:30.959311    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wr97s"
Oct 30 22:27:31.035123 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.035084    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"]
Oct 30 22:27:31.035323 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.035272    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr" podUID="5e3cc6b9-79e0-49d3-8403-65e42254365e" containerName="discovery" containerID="cri-o://856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c" gracePeriod=30
Oct 30 22:27:31.279371 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.279343    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:27:31.348759 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.348727    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-ca-configmap\") pod \"5e3cc6b9-79e0-49d3-8403-65e42254365e\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") "
Oct 30 22:27:31.348963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.348784    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-kubeconfig\") pod \"5e3cc6b9-79e0-49d3-8403-65e42254365e\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") "
Oct 30 22:27:31.348963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.348805    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-dns-cert\") pod \"5e3cc6b9-79e0-49d3-8403-65e42254365e\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") "
Oct 30 22:27:31.348963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.348856    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-token\") pod \"5e3cc6b9-79e0-49d3-8403-65e42254365e\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") "
Oct 30 22:27:31.348963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.348882    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-kube-api-access-22f7w\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-kube-api-access-22f7w\") pod \"5e3cc6b9-79e0-49d3-8403-65e42254365e\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") "
Oct 30 22:27:31.348963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.348944    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e3cc6b9-79e0-49d3-8403-65e42254365e-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cc6b9-79e0-49d3-8403-65e42254365e-local-certs\") pod \"5e3cc6b9-79e0-49d3-8403-65e42254365e\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") "
Oct 30 22:27:31.349216 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.348966    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-cacerts\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-cacerts\") pod \"5e3cc6b9-79e0-49d3-8403-65e42254365e\" (UID: \"5e3cc6b9-79e0-49d3-8403-65e42254365e\") "
Oct 30 22:27:31.349216 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.349166    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-ca-configmap" pod "5e3cc6b9-79e0-49d3-8403-65e42254365e" (UID: "5e3cc6b9-79e0-49d3-8403-65e42254365e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:27:31.351368 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.351334    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-dns-cert" pod "5e3cc6b9-79e0-49d3-8403-65e42254365e" (UID: "5e3cc6b9-79e0-49d3-8403-65e42254365e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:27:31.351494 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.351364    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-cacerts" pod "5e3cc6b9-79e0-49d3-8403-65e42254365e" (UID: "5e3cc6b9-79e0-49d3-8403-65e42254365e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:27:31.351741 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.351712    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-kube-api-access-22f7w" pod "5e3cc6b9-79e0-49d3-8403-65e42254365e" (UID: "5e3cc6b9-79e0-49d3-8403-65e42254365e"). InnerVolumeSpecName "kube-api-access-22f7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:31.351839 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.351752    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-token" pod "5e3cc6b9-79e0-49d3-8403-65e42254365e" (UID: "5e3cc6b9-79e0-49d3-8403-65e42254365e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:27:31.351839 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.351760    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e3cc6b9-79e0-49d3-8403-65e42254365e-local-certs" pod "5e3cc6b9-79e0-49d3-8403-65e42254365e" (UID: "5e3cc6b9-79e0-49d3-8403-65e42254365e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:27:31.351839 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.351760    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-kubeconfig" pod "5e3cc6b9-79e0-49d3-8403-65e42254365e" (UID: "5e3cc6b9-79e0-49d3-8403-65e42254365e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:27:31.450204 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.450155    5503 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-ca-configmap\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:31.450204 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.450198    5503 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-kubeconfig\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:31.450204 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.450208    5503 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-csr-dns-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:31.450204 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.450216    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:31.450204 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.450225    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22f7w\" (UniqueName: \"kubernetes.io/projected/5e3cc6b9-79e0-49d3-8403-65e42254365e-kube-api-access-22f7w\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:31.450508 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.450233    5503 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5e3cc6b9-79e0-49d3-8403-65e42254365e-local-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:31.450508 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.450241    5503 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5e3cc6b9-79e0-49d3-8403-65e42254365e-cacerts\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:27:31.965629 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.965588    5503 generic.go:356] "Generic (PLEG): container finished" podID="5e3cc6b9-79e0-49d3-8403-65e42254365e" containerID="856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c" exitCode=0
Oct 30 22:27:31.966114 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.965655    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"
Oct 30 22:27:31.966114 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.965660    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr" event={"ID":"5e3cc6b9-79e0-49d3-8403-65e42254365e","Type":"ContainerDied","Data":"856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c"}
Oct 30 22:27:31.966114 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.965743    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr" event={"ID":"5e3cc6b9-79e0-49d3-8403-65e42254365e","Type":"ContainerDied","Data":"2fd42b50895aa370effea9237309260ad7c2bc1aa9de0f7b1a08ef71edd46e41"}
Oct 30 22:27:31.966114 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.965769    5503 scope.go:122] "RemoveContainer" containerID="856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c"
Oct 30 22:27:31.974741 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.974696    5503 scope.go:122] "RemoveContainer" containerID="856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c"
Oct 30 22:27:31.975109 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:27:31.975089    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c\": container with ID starting with 856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c not found: ID does not exist" containerID="856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c"
Oct 30 22:27:31.975185 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.975119    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c"} err="failed to get container status \"856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c\": rpc error: code = NotFound desc = could not find container \"856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c\": container with ID starting with 856dbf9951c75c70b35d0772d89f0c3f864608147bdc2f3b6c3330488988137c not found: ID does not exist"
Oct 30 22:27:31.986400 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.986375    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"]
Oct 30 22:27:31.991172 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:31.991151    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr"]
Oct 30 22:27:33.855227 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:33.855189    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e3cc6b9-79e0-49d3-8403-65e42254365e" path="/var/lib/kubelet/pods/5e3cc6b9-79e0-49d3-8403-65e42254365e/volumes"
Oct 30 22:27:36.209482 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:36.209435    5503 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-ldwtr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.26:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 22:27:36.209921 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:36.209507    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-ldwtr" podUID="5e3cc6b9-79e0-49d3-8403-65e42254365e" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.26:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:27:45.389270 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:27:45.389230    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:27:46.122514 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.122477    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:46.225973 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.225941    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:46.244120 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.244082    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-76d8b77fbb-9rjtf"]
Oct 30 22:27:46.244514 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.244501    5503 cpu_manager.go:414] "containerMap: removing container" podUID="5e3cc6b9-79e0-49d3-8403-65e42254365e" containerName="discovery"
Oct 30 22:27:46.244555 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.244518    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e3cc6b9-79e0-49d3-8403-65e42254365e" containerName="discovery"
Oct 30 22:27:46.244596 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.244586    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e3cc6b9-79e0-49d3-8403-65e42254365e" containerName="discovery"
Oct 30 22:27:46.250638 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.250616    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:27:46.259042 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.259014    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-76d8b77fbb-9rjtf"]
Oct 30 22:27:46.260779 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.260748    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Oct 30 22:27:46.260888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.260798    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-2xx62\""
Oct 30 22:27:46.260888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.260820    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Oct 30 22:27:46.260888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.260831    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Oct 30 22:27:46.273590 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.273551    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"]
Oct 30 22:27:46.278029 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.278003    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:27:46.287194 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.287164    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"]
Oct 30 22:27:46.346317 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.346259    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:46.362241 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.362217    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-bnlq4\""
Oct 30 22:27:46.362367 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.362309    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\""
Oct 30 22:27:46.388693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.388576    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bacb935-8b35-4c52-8675-41c6bd60d1c8-cert\") pod \"kserve-controller-manager-76d8b77fbb-9rjtf\" (UID: \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:27:46.388693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.388642    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmvfn\" (UniqueName: \"kubernetes.io/projected/3bacb935-8b35-4c52-8675-41c6bd60d1c8-kube-api-access-wmvfn\") pod \"kserve-controller-manager-76d8b77fbb-9rjtf\" (UID: \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:27:46.388867 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.388827    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb9qw\" (UniqueName: \"kubernetes.io/projected/d165075d-a6ec-456a-9fa3-25f176eef900-kube-api-access-vb9qw\") pod \"llmisvc-controller-manager-86c86ffb6b-w7p6x\" (UID: \"d165075d-a6ec-456a-9fa3-25f176eef900\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:27:46.388867 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.388856    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d165075d-a6ec-456a-9fa3-25f176eef900-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-w7p6x\" (UID: \"d165075d-a6ec-456a-9fa3-25f176eef900\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:27:46.448949 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.448915    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:46.489653 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.489617    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d165075d-a6ec-456a-9fa3-25f176eef900-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-w7p6x\" (UID: \"d165075d-a6ec-456a-9fa3-25f176eef900\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:27:46.489805 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.489703    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bacb935-8b35-4c52-8675-41c6bd60d1c8-cert\") pod \"kserve-controller-manager-76d8b77fbb-9rjtf\" (UID: \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:27:46.489805 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.489753    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmvfn\" (UniqueName: \"kubernetes.io/projected/3bacb935-8b35-4c52-8675-41c6bd60d1c8-kube-api-access-wmvfn\") pod \"kserve-controller-manager-76d8b77fbb-9rjtf\" (UID: \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:27:46.489924 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.489819    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb9qw\" (UniqueName: \"kubernetes.io/projected/d165075d-a6ec-456a-9fa3-25f176eef900-kube-api-access-vb9qw\") pod \"llmisvc-controller-manager-86c86ffb6b-w7p6x\" (UID: \"d165075d-a6ec-456a-9fa3-25f176eef900\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:27:46.492359 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.492333    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d165075d-a6ec-456a-9fa3-25f176eef900-cert\") pod \"llmisvc-controller-manager-86c86ffb6b-w7p6x\" (UID: \"d165075d-a6ec-456a-9fa3-25f176eef900\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:27:46.492489 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.492468    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bacb935-8b35-4c52-8675-41c6bd60d1c8-cert\") pod \"kserve-controller-manager-76d8b77fbb-9rjtf\" (UID: \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:27:46.502895 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.502870    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmvfn\" (UniqueName: \"kubernetes.io/projected/3bacb935-8b35-4c52-8675-41c6bd60d1c8-kube-api-access-wmvfn\") pod \"kserve-controller-manager-76d8b77fbb-9rjtf\" (UID: \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:27:46.503022 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.503000    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb9qw\" (UniqueName: \"kubernetes.io/projected/d165075d-a6ec-456a-9fa3-25f176eef900-kube-api-access-vb9qw\") pod \"llmisvc-controller-manager-86c86ffb6b-w7p6x\" (UID: \"d165075d-a6ec-456a-9fa3-25f176eef900\") " pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:27:46.565753 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.565714    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:27:46.591897 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.591863    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:27:46.843139 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.842957    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-76d8b77fbb-9rjtf"]
Oct 30 22:27:46.845661 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:27:46.845634    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bacb935_8b35_4c52_8675_41c6bd60d1c8.slice/crio-63528c80af663c13ca5cac46b0e95bf0139f9fdd957293cfca94683311d1860c WatchSource:0}: Error finding container 63528c80af663c13ca5cac46b0e95bf0139f9fdd957293cfca94683311d1860c: Status 404 returned error can't find the container with id 63528c80af663c13ca5cac46b0e95bf0139f9fdd957293cfca94683311d1860c
Oct 30 22:27:46.875462 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:46.875437    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"]
Oct 30 22:27:46.877338 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:27:46.877294    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd165075d_a6ec_456a_9fa3_25f176eef900.slice/crio-412bb5c92a4ccf7f18a2ea993a18527d32e217bd998b7accecc1543afcca211d WatchSource:0}: Error finding container 412bb5c92a4ccf7f18a2ea993a18527d32e217bd998b7accecc1543afcca211d: Status 404 returned error can't find the container with id 412bb5c92a4ccf7f18a2ea993a18527d32e217bd998b7accecc1543afcca211d
Oct 30 22:27:47.049127 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:47.049043    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf" event={"ID":"3bacb935-8b35-4c52-8675-41c6bd60d1c8","Type":"ContainerStarted","Data":"63528c80af663c13ca5cac46b0e95bf0139f9fdd957293cfca94683311d1860c"}
Oct 30 22:27:47.050267 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:47.050243    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x" event={"ID":"d165075d-a6ec-456a-9fa3-25f176eef900","Type":"ContainerStarted","Data":"412bb5c92a4ccf7f18a2ea993a18527d32e217bd998b7accecc1543afcca211d"}
Oct 30 22:27:47.744171 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:47.744123    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:47.855161 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:47.855126    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:47.969010 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:47.963901    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:49.220961 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:49.220923    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:27:51.075680 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:51.075559    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x" event={"ID":"d165075d-a6ec-456a-9fa3-25f176eef900","Type":"ContainerStarted","Data":"bb639634ede2b01bea9bbb3006d6d821b8f3473046148016cbe10ebcbf4e0480"}
Oct 30 22:27:51.076090 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:51.075704    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:27:51.077047 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:51.077022    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf" event={"ID":"3bacb935-8b35-4c52-8675-41c6bd60d1c8","Type":"ContainerStarted","Data":"454b86c826a4281004abd15a26ca752dcdd97785d69c261bed3d2bcdd4726c13"}
Oct 30 22:27:51.077174 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:51.077160    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:27:51.096897 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:51.096827    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x" podStartSLOduration=1.193774249 podStartE2EDuration="5.096810321s" totalImagesPullingTime="3.903036072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:27:51.095943264 +0000 UTC m=+513.365024787" watchObservedRunningTime="2026-10-30 22:27:51.096810321 +0000 UTC m=+513.365891846"
Oct 30 22:27:51.116514 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:27:51.116461    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf" podStartSLOduration=1.223364918 podStartE2EDuration="5.116444354s" totalImagesPullingTime="3.893079436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:27:51.115517663 +0000 UTC m=+513.384599186" watchObservedRunningTime="2026-10-30 22:27:51.116444354 +0000 UTC m=+513.385525879"
Oct 30 22:28:02.392011 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:02.391975    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:02.492028 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:02.491997    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:09.133410 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:09.133376    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:28:15.389444 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:28:15.389401    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:28:22.083182 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:22.083143    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-86c86ffb6b-w7p6x"
Oct 30 22:28:22.085888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:22.085870    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:28:34.058846 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.058765    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-76d8b77fbb-9rjtf"]
Oct 30 22:28:34.059286 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.059056    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf" podUID="3bacb935-8b35-4c52-8675-41c6bd60d1c8" containerName="manager" containerID="cri-o://454b86c826a4281004abd15a26ca752dcdd97785d69c261bed3d2bcdd4726c13" gracePeriod=10
Oct 30 22:28:34.085083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.085049    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-76d8b77fbb-9nwtn"]
Oct 30 22:28:34.087734 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.087706    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:28:34.099170 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.099138    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-76d8b77fbb-9nwtn"]
Oct 30 22:28:34.130545 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.130518    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2htwb\" (UniqueName: \"kubernetes.io/projected/395cbd9e-c968-42ef-a3a4-ec39e0e6fceb-kube-api-access-2htwb\") pod \"kserve-controller-manager-76d8b77fbb-9nwtn\" (UID: \"395cbd9e-c968-42ef-a3a4-ec39e0e6fceb\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:28:34.130722 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.130554    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/395cbd9e-c968-42ef-a3a4-ec39e0e6fceb-cert\") pod \"kserve-controller-manager-76d8b77fbb-9nwtn\" (UID: \"395cbd9e-c968-42ef-a3a4-ec39e0e6fceb\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:28:34.231481 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.231444    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2htwb\" (UniqueName: \"kubernetes.io/projected/395cbd9e-c968-42ef-a3a4-ec39e0e6fceb-kube-api-access-2htwb\") pod \"kserve-controller-manager-76d8b77fbb-9nwtn\" (UID: \"395cbd9e-c968-42ef-a3a4-ec39e0e6fceb\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:28:34.231615 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.231505    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/395cbd9e-c968-42ef-a3a4-ec39e0e6fceb-cert\") pod \"kserve-controller-manager-76d8b77fbb-9nwtn\" (UID: \"395cbd9e-c968-42ef-a3a4-ec39e0e6fceb\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:28:34.234349 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.234317    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/395cbd9e-c968-42ef-a3a4-ec39e0e6fceb-cert\") pod \"kserve-controller-manager-76d8b77fbb-9nwtn\" (UID: \"395cbd9e-c968-42ef-a3a4-ec39e0e6fceb\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:28:34.245152 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.245120    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2htwb\" (UniqueName: \"kubernetes.io/projected/395cbd9e-c968-42ef-a3a4-ec39e0e6fceb-kube-api-access-2htwb\") pod \"kserve-controller-manager-76d8b77fbb-9nwtn\" (UID: \"395cbd9e-c968-42ef-a3a4-ec39e0e6fceb\") " pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:28:34.287190 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.287160    5503 generic.go:356] "Generic (PLEG): container finished" podID="3bacb935-8b35-4c52-8675-41c6bd60d1c8" containerID="454b86c826a4281004abd15a26ca752dcdd97785d69c261bed3d2bcdd4726c13" exitCode=0
Oct 30 22:28:34.287313 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.287227    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf" event={"ID":"3bacb935-8b35-4c52-8675-41c6bd60d1c8","Type":"ContainerDied","Data":"454b86c826a4281004abd15a26ca752dcdd97785d69c261bed3d2bcdd4726c13"}
Oct 30 22:28:34.304144 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.304120    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:28:34.434692 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.434631    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3bacb935-8b35-4c52-8675-41c6bd60d1c8-cert\" (UniqueName: \"kubernetes.io/secret/3bacb935-8b35-4c52-8675-41c6bd60d1c8-cert\") pod \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\" (UID: \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\") "
Oct 30 22:28:34.434891 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.434741    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3bacb935-8b35-4c52-8675-41c6bd60d1c8-kube-api-access-wmvfn\" (UniqueName: \"kubernetes.io/projected/3bacb935-8b35-4c52-8675-41c6bd60d1c8-kube-api-access-wmvfn\") pod \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\" (UID: \"3bacb935-8b35-4c52-8675-41c6bd60d1c8\") "
Oct 30 22:28:34.437021 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.436990    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bacb935-8b35-4c52-8675-41c6bd60d1c8-cert" pod "3bacb935-8b35-4c52-8675-41c6bd60d1c8" (UID: "3bacb935-8b35-4c52-8675-41c6bd60d1c8"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:28:34.437133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.437038    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bacb935-8b35-4c52-8675-41c6bd60d1c8-kube-api-access-wmvfn" pod "3bacb935-8b35-4c52-8675-41c6bd60d1c8" (UID: "3bacb935-8b35-4c52-8675-41c6bd60d1c8"). InnerVolumeSpecName "kube-api-access-wmvfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:28:34.449888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.449861    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:28:34.535337 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.535304    5503 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bacb935-8b35-4c52-8675-41c6bd60d1c8-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:34.535337 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.535336    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmvfn\" (UniqueName: \"kubernetes.io/projected/3bacb935-8b35-4c52-8675-41c6bd60d1c8-kube-api-access-wmvfn\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:28:34.722917 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:34.722891    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-76d8b77fbb-9nwtn"]
Oct 30 22:28:34.725161 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:28:34.725136    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395cbd9e_c968_42ef_a3a4_ec39e0e6fceb.slice/crio-a024a67a40606f95bbc99f457cef656b413139c0c559e6c5e30e5768543c9e65 WatchSource:0}: Error finding container a024a67a40606f95bbc99f457cef656b413139c0c559e6c5e30e5768543c9e65: Status 404 returned error can't find the container with id a024a67a40606f95bbc99f457cef656b413139c0c559e6c5e30e5768543c9e65
Oct 30 22:28:35.293069 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.293036    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf"
Oct 30 22:28:35.293507 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.293072    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-76d8b77fbb-9rjtf" event={"ID":"3bacb935-8b35-4c52-8675-41c6bd60d1c8","Type":"ContainerDied","Data":"63528c80af663c13ca5cac46b0e95bf0139f9fdd957293cfca94683311d1860c"}
Oct 30 22:28:35.293507 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.293142    5503 scope.go:122] "RemoveContainer" containerID="454b86c826a4281004abd15a26ca752dcdd97785d69c261bed3d2bcdd4726c13"
Oct 30 22:28:35.294845 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.294727    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn" event={"ID":"395cbd9e-c968-42ef-a3a4-ec39e0e6fceb","Type":"ContainerStarted","Data":"8aa6570a1356b0a684b88db12962caac614c8065db5936284cb12c645c273cad"}
Oct 30 22:28:35.294845 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.294777    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn" event={"ID":"395cbd9e-c968-42ef-a3a4-ec39e0e6fceb","Type":"ContainerStarted","Data":"a024a67a40606f95bbc99f457cef656b413139c0c559e6c5e30e5768543c9e65"}
Oct 30 22:28:35.294845 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.294818    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:28:35.318740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.318680    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn" podStartSLOduration=0.858326757 podStartE2EDuration="1.31865041s" totalImagesPullingTime="460.323653ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:28:35.316633816 +0000 UTC m=+557.585715340" watchObservedRunningTime="2026-10-30 22:28:35.31865041 +0000 UTC m=+557.587731932"
Oct 30 22:28:35.332006 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.331919    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-76d8b77fbb-9rjtf"]
Oct 30 22:28:35.335252 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.335214    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-76d8b77fbb-9rjtf"]
Oct 30 22:28:35.854944 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:35.854915    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3bacb935-8b35-4c52-8675-41c6bd60d1c8" path="/var/lib/kubelet/pods/3bacb935-8b35-4c52-8675-41c6bd60d1c8/volumes"
Oct 30 22:28:45.389428 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:28:45.389388    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:28:49.326505 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:28:49.326446    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:06.306258 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:06.306224    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-76d8b77fbb-9nwtn"
Oct 30 22:29:08.156890 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.156858    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:08.262553 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.262520    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:08.337112 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.337076    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-nq8qs"]
Oct 30 22:29:08.337476 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.337462    5503 cpu_manager.go:414] "containerMap: removing container" podUID="3bacb935-8b35-4c52-8675-41c6bd60d1c8" containerName="manager"
Oct 30 22:29:08.337519 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.337478    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3bacb935-8b35-4c52-8675-41c6bd60d1c8" containerName="manager"
Oct 30 22:29:08.337552 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.337546    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="3bacb935-8b35-4c52-8675-41c6bd60d1c8" containerName="manager"
Oct 30 22:29:08.340554 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.340531    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.352490 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.352462    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-nq8qs"]
Oct 30 22:29:08.370380 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.370346    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:08.404400 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.404365    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\""
Oct 30 22:29:08.404866 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.404850    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-7lrzk\""
Oct 30 22:29:08.405081 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.405065    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-metrics-tls\""
Oct 30 22:29:08.426496 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.426415    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6718fe7a-ba3b-4e93-affc-3ebd5a036649-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-nq8qs\" (UID: \"6718fe7a-ba3b-4e93-affc-3ebd5a036649\") " pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.426496 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.426478    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6718fe7a-ba3b-4e93-affc-3ebd5a036649-cert\") pod \"odh-model-controller-7f9f8cdc65-nq8qs\" (UID: \"6718fe7a-ba3b-4e93-affc-3ebd5a036649\") " pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.426689 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.426566    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjjp4\" (UniqueName: \"kubernetes.io/projected/6718fe7a-ba3b-4e93-affc-3ebd5a036649-kube-api-access-hjjp4\") pod \"odh-model-controller-7f9f8cdc65-nq8qs\" (UID: \"6718fe7a-ba3b-4e93-affc-3ebd5a036649\") " pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.470446 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.470417    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:08.527549 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.527516    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6718fe7a-ba3b-4e93-affc-3ebd5a036649-cert\") pod \"odh-model-controller-7f9f8cdc65-nq8qs\" (UID: \"6718fe7a-ba3b-4e93-affc-3ebd5a036649\") " pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.527768 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.527556    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjjp4\" (UniqueName: \"kubernetes.io/projected/6718fe7a-ba3b-4e93-affc-3ebd5a036649-kube-api-access-hjjp4\") pod \"odh-model-controller-7f9f8cdc65-nq8qs\" (UID: \"6718fe7a-ba3b-4e93-affc-3ebd5a036649\") " pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.527768 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.527607    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6718fe7a-ba3b-4e93-affc-3ebd5a036649-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-nq8qs\" (UID: \"6718fe7a-ba3b-4e93-affc-3ebd5a036649\") " pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.530319 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.530293    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6718fe7a-ba3b-4e93-affc-3ebd5a036649-metrics-certs\") pod \"odh-model-controller-7f9f8cdc65-nq8qs\" (UID: \"6718fe7a-ba3b-4e93-affc-3ebd5a036649\") " pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.530395 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.530293    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6718fe7a-ba3b-4e93-affc-3ebd5a036649-cert\") pod \"odh-model-controller-7f9f8cdc65-nq8qs\" (UID: \"6718fe7a-ba3b-4e93-affc-3ebd5a036649\") " pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.540805 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.540781    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjjp4\" (UniqueName: \"kubernetes.io/projected/6718fe7a-ba3b-4e93-affc-3ebd5a036649-kube-api-access-hjjp4\") pod \"odh-model-controller-7f9f8cdc65-nq8qs\" (UID: \"6718fe7a-ba3b-4e93-affc-3ebd5a036649\") " pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.653519 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.653469    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:08.929755 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:08.929726    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-7f9f8cdc65-nq8qs"]
Oct 30 22:29:08.932404 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:29:08.932373    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6718fe7a_ba3b_4e93_affc_3ebd5a036649.slice/crio-79291c6c5e58091ad6936cba7842877a23dd99f8c2546ebd45e1ee977af9838d WatchSource:0}: Error finding container 79291c6c5e58091ad6936cba7842877a23dd99f8c2546ebd45e1ee977af9838d: Status 404 returned error can't find the container with id 79291c6c5e58091ad6936cba7842877a23dd99f8c2546ebd45e1ee977af9838d
Oct 30 22:29:09.183895 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:09.183865    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:09.460075 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:09.459983    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs" event={"ID":"6718fe7a-ba3b-4e93-affc-3ebd5a036649","Type":"ContainerStarted","Data":"79291c6c5e58091ad6936cba7842877a23dd99f8c2546ebd45e1ee977af9838d"}
Oct 30 22:29:11.554118 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:11.554076    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:11.656378 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:11.656341    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:12.480544 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:12.480510    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs" event={"ID":"6718fe7a-ba3b-4e93-affc-3ebd5a036649","Type":"ContainerStarted","Data":"da8bcfa7861de90985f77002adbd6308705e4941aa60c8e812dadf387143debb"}
Oct 30 22:29:12.480782 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:12.480617    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:12.501866 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:12.501814    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs" podStartSLOduration=1.667103698 podStartE2EDuration="4.501799549s" totalImagesPullingTime="2.834695851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:29:12.500270205 +0000 UTC m=+594.769351732" watchObservedRunningTime="2026-10-30 22:29:12.501799549 +0000 UTC m=+594.770881072"
Oct 30 22:29:15.396936 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:29:15.396898    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:29:23.488167 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:23.488133    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-7f9f8cdc65-nq8qs"
Oct 30 22:29:37.308352 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:37.308311    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:38.796399 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:38.796348    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:29:45.390373 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:29:45.390314    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:29:59.373685 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:29:59.373631    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:09.230151 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:30:09.230120    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:15.390192 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:30:15.390153    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:30:23.095171 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:30:23.095134    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:45.389682 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:30:45.389624    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:30:57.017847 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:30:57.017797    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:57.134046 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:30:57.134002    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:30:59.409802 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:30:59.409769    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:04.256211 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:04.256182    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:15.390149 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:31:15.390108    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:31:21.559495 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:21.559444    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:21.761341 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:21.761303    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:21.874462 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:21.874428    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:25.204794 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:25.204737    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:26.504218 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.504177    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:26.842439 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.842374    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"]
Oct 30 22:31:26.845995 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.845973    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:31:26.855649 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.855623    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"openshift-service-ca.crt\""
Oct 30 22:31:26.855862 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.855837    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"default-dockercfg-q6gwn\""
Oct 30 22:31:26.855862 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.855839    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-kserve-self-signed-certs\""
Oct 30 22:31:26.855993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.855892    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"scheduler-inline-config-test-epp-sa-dockercfg-r6nbv\""
Oct 30 22:31:26.855993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.855897    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-47655ff5\"/\"kube-root-ca.crt\""
Oct 30 22:31:26.857499 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.857480    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"]
Oct 30 22:31:26.887536 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.887502    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:26.967002 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.966959    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w\" (UID: \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:31:26.967002 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:26.967003    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd4df\" (UniqueName: \"kubernetes.io/projected/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-kube-api-access-jd4df\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w\" (UID: \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:31:27.068049 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:27.068012    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w\" (UID: \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:31:27.068049 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:27.068055    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd4df\" (UniqueName: \"kubernetes.io/projected/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-kube-api-access-jd4df\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w\" (UID: \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:31:27.070627 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:27.070599    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-tls-certs\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w\" (UID: \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:31:27.081254 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:27.081230    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd4df\" (UniqueName: \"kubernetes.io/projected/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-kube-api-access-jd4df\") pod \"scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w\" (UID: \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\") " pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:31:27.159433 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:27.159345    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:31:27.458150 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:27.458124    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"]
Oct 30 22:31:27.460291 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:31:27.460265    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e06e66_95d4_476b_b5a5_10e4ba8ced4e.slice/crio-955a6a62e09a581b5a8f89d7ef3a7adac9127f1e5b44b7fccc32b5afa33d902b WatchSource:0}: Error finding container 955a6a62e09a581b5a8f89d7ef3a7adac9127f1e5b44b7fccc32b5afa33d902b: Status 404 returned error can't find the container with id 955a6a62e09a581b5a8f89d7ef3a7adac9127f1e5b44b7fccc32b5afa33d902b
Oct 30 22:31:27.462247 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:27.462228    5503 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:31:28.144446 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:28.144403    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" event={"ID":"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e","Type":"ContainerStarted","Data":"955a6a62e09a581b5a8f89d7ef3a7adac9127f1e5b44b7fccc32b5afa33d902b"}
Oct 30 22:31:30.156811 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:30.156779    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" event={"ID":"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e","Type":"ContainerStarted","Data":"1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746"}
Oct 30 22:31:30.157299 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:30.156841    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:31:30.176950 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:30.176889    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" podStartSLOduration=2.304438163 podStartE2EDuration="4.176871618s" totalImagesPullingTime="1.872433455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:31:30.176823221 +0000 UTC m=+732.445904746" watchObservedRunningTime="2026-10-30 22:31:30.176871618 +0000 UTC m=+732.445953139"
Oct 30 22:31:37.194379 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:37.194348    5503 generic.go:356] "Generic (PLEG): container finished" podID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerID="e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f" exitCode=0
Oct 30 22:31:37.194884 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:37.194424    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" event={"ID":"c2fe0161-5b10-4d97-97f4-af020ffc5d78","Type":"ContainerDied","Data":"e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f"}
Oct 30 22:31:37.194884 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:37.194837    5503 scope.go:122] "RemoveContainer" containerID="e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f"
Oct 30 22:31:37.195943 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:37.195919    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:31:37.196016 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:37.195992    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:31:37.196067 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:37.196030    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:31:38.229191 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:38.229149    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" event={"ID":"c2fe0161-5b10-4d97-97f4-af020ffc5d78","Type":"ContainerStarted","Data":"d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e"}
Oct 30 22:31:39.114083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:39.114047    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:31:39.116984 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:39.116953    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-4wfkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Oct 30 22:31:39.117233 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:39.117200    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Oct 30 22:31:40.114103 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:40.114064    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-4wfkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Oct 30 22:31:40.115408 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:40.115373    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Oct 30 22:31:40.219594 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:40.219557    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:41.113142 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:41.113104    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-4wfkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Oct 30 22:31:41.113416 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:41.113391    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Oct 30 22:31:41.242037 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:41.242003    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:31:42.112694 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:42.104152    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-4wfkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body=
Oct 30 22:31:42.112694 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:42.104214    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused"
Oct 30 22:31:43.106774 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:43.106734    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:31:43.108183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:43.108157    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 22:31:45.225633 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:31:45.225591    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:31:45.391795 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:31:45.391749    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:32:01.164101 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:01.164062    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:32:02.511930 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:02.511895    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"]
Oct 30 22:32:02.523591 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:02.523553    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:02.646373 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:02.646337    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:04.303291 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:04.303262    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:11.163796 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:11.163759    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" podUID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:32:15.389119 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:32:15.389084    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:32:17.534027 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:17.533963    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" podUID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" containerName="main" containerID="cri-o://1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746" gracePeriod=45
Oct 30 22:32:21.163541 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:21.163499    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" podUID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:32:31.164160 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:31.164113    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" podUID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:32:31.164638 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:31.164257    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:32:41.163710 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:41.163655    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" podUID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:32:45.390764 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:32:45.390723    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:32:47.656119 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:32:47.656075    5503 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e06e66_95d4_476b_b5a5_10e4ba8ced4e.slice/crio-1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e06e66_95d4_476b_b5a5_10e4ba8ced4e.slice/crio-conmon-1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:32:47.656397 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:32:47.656232    5503 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e06e66_95d4_476b_b5a5_10e4ba8ced4e.slice/crio-conmon-1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 22:32:47.788918 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:47.788892    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:32:47.904352 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:47.904328    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-kube-api-access-jd4df\" (UniqueName: \"kubernetes.io/projected/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-kube-api-access-jd4df\") pod \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\" (UID: \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\") "
Oct 30 22:32:47.904472 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:47.904387    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-tls-certs\") pod \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\" (UID: \"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e\") "
Oct 30 22:32:47.906573 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:47.906545    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-tls-certs" pod "b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" (UID: "b4e06e66-95d4-476b-b5a5-10e4ba8ced4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:32:47.906709 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:47.906604    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-kube-api-access-jd4df" pod "b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" (UID: "b4e06e66-95d4-476b-b5a5-10e4ba8ced4e"). InnerVolumeSpecName "kube-api-access-jd4df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:32:48.005634 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.005607    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd4df\" (UniqueName: \"kubernetes.io/projected/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-kube-api-access-jd4df\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:48.005634 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.005633    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:32:48.559784 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.559755    5503 generic.go:356] "Generic (PLEG): container finished" podID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" containerID="1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746" exitCode=0
Oct 30 22:32:48.559942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.559798    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"
Oct 30 22:32:48.559942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.559843    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" event={"ID":"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e","Type":"ContainerDied","Data":"1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746"}
Oct 30 22:32:48.559942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.559899    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w" event={"ID":"b4e06e66-95d4-476b-b5a5-10e4ba8ced4e","Type":"ContainerDied","Data":"955a6a62e09a581b5a8f89d7ef3a7adac9127f1e5b44b7fccc32b5afa33d902b"}
Oct 30 22:32:48.559942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.559927    5503 scope.go:122] "RemoveContainer" containerID="1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746"
Oct 30 22:32:48.569288 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.569271    5503 scope.go:122] "RemoveContainer" containerID="1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746"
Oct 30 22:32:48.569523 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:32:48.569507    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746\": container with ID starting with 1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746 not found: ID does not exist" containerID="1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746"
Oct 30 22:32:48.569566 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.569533    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746"} err="failed to get container status \"1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746\": rpc error: code = NotFound desc = could not find container \"1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746\": container with ID starting with 1e1f17f0b20f3dd9865a17076d37bd18af4a63a7293f722c132137e8c1913746 not found: ID does not exist"
Oct 30 22:32:48.582331 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.582312    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"]
Oct 30 22:32:48.586464 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:48.586443    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-47655ff5/scheduler-inline-config-test-kserve-router-scheduler-d859bbw26w"]
Oct 30 22:32:49.855210 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:49.855170    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" path="/var/lib/kubelet/pods/b4e06e66-95d4-476b-b5a5-10e4ba8ced4e/volumes"
Oct 30 22:32:52.763185 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:52.763155    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:52.863408 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:52.863385    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:54.546313 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.546285    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:54.762022 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.761987    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"]
Oct 30 22:32:54.762489 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.762462    5503 cpu_manager.go:414] "containerMap: removing container" podUID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" containerName="main"
Oct 30 22:32:54.762489 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.762484    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" containerName="main"
Oct 30 22:32:54.762720 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.762605    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="b4e06e66-95d4-476b-b5a5-10e4ba8ced4e" containerName="main"
Oct 30 22:32:54.767238 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.767214    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:32:54.775117 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.775098    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"kube-root-ca.crt\""
Oct 30 22:32:54.775331 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.775311    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisvc-model-qwen2-5-0-5b-rout-4c1ba212-epp-sa-dockercfg-khwvs\""
Oct 30 22:32:54.775451 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.775168    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"default-dockercfg-qnqtx\""
Oct 30 22:32:54.775451 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.775438    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"llmisv431d7e30f13554296364403a5b6365f5-kserve-self-signed-certs\""
Oct 30 22:32:54.775624 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.775225    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-aae51b37\"/\"openshift-service-ca.crt\""
Oct 30 22:32:54.776861 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.776831    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"]
Oct 30 22:32:54.805860 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.805734    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:54.854032 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.854004    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/777b6c69-33b0-4454-9f22-8055e33104a9-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx\" (UID: \"777b6c69-33b0-4454-9f22-8055e33104a9\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:32:54.854133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.854058    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8jqk\" (UniqueName: \"kubernetes.io/projected/777b6c69-33b0-4454-9f22-8055e33104a9-kube-api-access-r8jqk\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx\" (UID: \"777b6c69-33b0-4454-9f22-8055e33104a9\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:32:54.954941 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.954903    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/777b6c69-33b0-4454-9f22-8055e33104a9-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx\" (UID: \"777b6c69-33b0-4454-9f22-8055e33104a9\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:32:54.955046 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.954957    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8jqk\" (UniqueName: \"kubernetes.io/projected/777b6c69-33b0-4454-9f22-8055e33104a9-kube-api-access-r8jqk\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx\" (UID: \"777b6c69-33b0-4454-9f22-8055e33104a9\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:32:54.957501 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.957478    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/777b6c69-33b0-4454-9f22-8055e33104a9-tls-certs\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx\" (UID: \"777b6c69-33b0-4454-9f22-8055e33104a9\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:32:54.969814 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:54.969790    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8jqk\" (UniqueName: \"kubernetes.io/projected/777b6c69-33b0-4454-9f22-8055e33104a9-kube-api-access-r8jqk\") pod \"llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx\" (UID: \"777b6c69-33b0-4454-9f22-8055e33104a9\") " pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:32:55.099492 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:55.099469    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:32:55.377147 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:55.377120    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"]
Oct 30 22:32:55.380014 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:32:55.379987    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777b6c69_33b0_4454_9f22_8055e33104a9.slice/crio-8cb38d13876341068c6dd318f7d082b4aedcbfcb5c595dc18f36b3ff18efd711 WatchSource:0}: Error finding container 8cb38d13876341068c6dd318f7d082b4aedcbfcb5c595dc18f36b3ff18efd711: Status 404 returned error can't find the container with id 8cb38d13876341068c6dd318f7d082b4aedcbfcb5c595dc18f36b3ff18efd711
Oct 30 22:32:55.626716 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:55.624829    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" event={"ID":"777b6c69-33b0-4454-9f22-8055e33104a9","Type":"ContainerStarted","Data":"d3d3afa03d13341b7f42cc428bee7d2e515c4d3eda5774dcd1845c2307aaeabf"}
Oct 30 22:32:55.626716 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:55.624943    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" event={"ID":"777b6c69-33b0-4454-9f22-8055e33104a9","Type":"ContainerStarted","Data":"8cb38d13876341068c6dd318f7d082b4aedcbfcb5c595dc18f36b3ff18efd711"}
Oct 30 22:32:55.626716 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:55.625872    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:32:55.646224 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:55.646134    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" podStartSLOduration=1.6461238630000001 podStartE2EDuration="1.646123863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:32:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:32:55.644451531 +0000 UTC m=+817.913533052" watchObservedRunningTime="2026-10-30 22:32:55.646123863 +0000 UTC m=+817.915205386"
Oct 30 22:32:59.177056 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:59.177018    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:32:59.277937 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:32:59.277903    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:04.471019 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:04.470994    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:05.040911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:05.040887    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:05.141706 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:05.141660    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:15.389150 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:33:15.389115    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:33:19.315575 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:19.315546    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:23.332711 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:23.332651    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:23.525362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:23.525329    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:23.648942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:23.648922    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:27.637119 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:27.637092    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:33:29.824631 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:29.824564    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"]
Oct 30 22:33:29.838869 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:29.838837    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:29.942269 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:29.942232    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:30.089555 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:30.089487    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:30.189326 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:30.189297    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:35.005982 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:35.005950    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:36.387962 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.387933    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"]
Oct 30 22:33:36.391227 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.391209    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.408299 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.408275    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"]
Oct 30 22:33:36.587906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.587872    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.588059 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.587922    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.588059 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.587976    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.588059 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.587991    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmmmk\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-kube-api-access-xmmmk\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.588236 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.588121    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.588236 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.588155    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.588316 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.588227    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.588316 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.588253    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b80c335-71f6-4e1c-acc5-fb6003572f92-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.588316 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.588269    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689266    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689309    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689339    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689365    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b80c335-71f6-4e1c-acc5-fb6003572f92-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689387    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689433    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689458    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689488    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689511    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmmmk\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-kube-api-access-xmmmk\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689957 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689835    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.689957 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689920    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.690028 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.689975    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.690500 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.690470    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b80c335-71f6-4e1c-acc5-fb6003572f92-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.690613 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.690482    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.691767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.691749    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.692024 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.692007    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.702748 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.702721    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.703049 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.703029    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmmmk\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-kube-api-access-xmmmk\") pod \"openshift-ai-inference-openshift-default-86b669b864-rb29g\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.703707 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.703688    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:36.988727 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.988701    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"]
Oct 30 22:33:36.991099 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:33:36.991075    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b80c335_71f6_4e1c_acc5_fb6003572f92.slice/crio-7ca7c42d9a48e9266712a20333a6516dcdb99ff93452908698711c3b11d4ba53 WatchSource:0}: Error finding container 7ca7c42d9a48e9266712a20333a6516dcdb99ff93452908698711c3b11d4ba53: Status 404 returned error can't find the container with id 7ca7c42d9a48e9266712a20333a6516dcdb99ff93452908698711c3b11d4ba53
Oct 30 22:33:36.992964 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.992934    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:33:36.993074 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.993008    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:33:36.993074 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:36.993049    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:33:37.637421 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:37.637385    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" podUID="777b6c69-33b0-4454-9f22-8055e33104a9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:33:37.833486 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:37.833419    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" event={"ID":"8b80c335-71f6-4e1c-acc5-fb6003572f92","Type":"ContainerStarted","Data":"3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5"}
Oct 30 22:33:37.833486 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:37.833486    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" event={"ID":"8b80c335-71f6-4e1c-acc5-fb6003572f92","Type":"ContainerStarted","Data":"7ca7c42d9a48e9266712a20333a6516dcdb99ff93452908698711c3b11d4ba53"}
Oct 30 22:33:37.860301 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:37.860247    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" podStartSLOduration=1.860229029 podStartE2EDuration="1.860229029s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:37.85666052 +0000 UTC m=+860.125742043" watchObservedRunningTime="2026-10-30 22:33:37.860229029 +0000 UTC m=+860.129310551"
Oct 30 22:33:38.703870 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:38.703834    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:38.708985 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:38.708959    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:38.838548 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:38.838517    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:38.839518 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:38.839496    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:33:40.110193 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:40.110157    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:44.844589 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:44.844538    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" podUID="777b6c69-33b0-4454-9f22-8055e33104a9" containerName="main" containerID="cri-o://d3d3afa03d13341b7f42cc428bee7d2e515c4d3eda5774dcd1845c2307aaeabf" gracePeriod=45
Oct 30 22:33:45.389251 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:33:45.389215    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:33:47.638377 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:47.638339    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" podUID="777b6c69-33b0-4454-9f22-8055e33104a9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:33:57.346064 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:57.346023    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:33:57.638077 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:57.637995    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" podUID="777b6c69-33b0-4454-9f22-8055e33104a9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:33:57.638227 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:33:57.638093    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:34:04.518816 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:04.518759    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:07.638044 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:07.638005    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" podUID="777b6c69-33b0-4454-9f22-8055e33104a9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:34:09.405981 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:09.405945    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:15.014403 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:15.014375    5503 generic.go:356] "Generic (PLEG): container finished" podID="777b6c69-33b0-4454-9f22-8055e33104a9" containerID="d3d3afa03d13341b7f42cc428bee7d2e515c4d3eda5774dcd1845c2307aaeabf" exitCode=0
Oct 30 22:34:15.014822 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:15.014455    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" event={"ID":"777b6c69-33b0-4454-9f22-8055e33104a9","Type":"ContainerDied","Data":"d3d3afa03d13341b7f42cc428bee7d2e515c4d3eda5774dcd1845c2307aaeabf"}
Oct 30 22:34:15.099108 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:15.099083    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:34:15.240615 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:15.240528    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/777b6c69-33b0-4454-9f22-8055e33104a9-kube-api-access-r8jqk\" (UniqueName: \"kubernetes.io/projected/777b6c69-33b0-4454-9f22-8055e33104a9-kube-api-access-r8jqk\") pod \"777b6c69-33b0-4454-9f22-8055e33104a9\" (UID: \"777b6c69-33b0-4454-9f22-8055e33104a9\") "
Oct 30 22:34:15.240615 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:15.240605    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/777b6c69-33b0-4454-9f22-8055e33104a9-tls-certs\" (UniqueName: \"kubernetes.io/secret/777b6c69-33b0-4454-9f22-8055e33104a9-tls-certs\") pod \"777b6c69-33b0-4454-9f22-8055e33104a9\" (UID: \"777b6c69-33b0-4454-9f22-8055e33104a9\") "
Oct 30 22:34:15.242977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:15.242937    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/777b6c69-33b0-4454-9f22-8055e33104a9-kube-api-access-r8jqk" pod "777b6c69-33b0-4454-9f22-8055e33104a9" (UID: "777b6c69-33b0-4454-9f22-8055e33104a9"). InnerVolumeSpecName "kube-api-access-r8jqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:34:15.242977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:15.242967    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/777b6c69-33b0-4454-9f22-8055e33104a9-tls-certs" pod "777b6c69-33b0-4454-9f22-8055e33104a9" (UID: "777b6c69-33b0-4454-9f22-8055e33104a9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:34:15.342149 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:15.342110    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8jqk\" (UniqueName: \"kubernetes.io/projected/777b6c69-33b0-4454-9f22-8055e33104a9-kube-api-access-r8jqk\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:15.342149 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:15.342143    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/777b6c69-33b0-4454-9f22-8055e33104a9-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:34:15.390255 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:34:15.390220    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:34:16.020277 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:16.020251    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"
Oct 30 22:34:16.020762 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:16.020254    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx" event={"ID":"777b6c69-33b0-4454-9f22-8055e33104a9","Type":"ContainerDied","Data":"8cb38d13876341068c6dd318f7d082b4aedcbfcb5c595dc18f36b3ff18efd711"}
Oct 30 22:34:16.020762 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:16.020379    5503 scope.go:122] "RemoveContainer" containerID="d3d3afa03d13341b7f42cc428bee7d2e515c4d3eda5774dcd1845c2307aaeabf"
Oct 30 22:34:16.039539 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:16.039510    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"]
Oct 30 22:34:16.048285 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:16.048260    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-aae51b37/llmisvc431d7e30f13554296364403a5b6365f5-kserve-router-schemrrdx"]
Oct 30 22:34:17.855601 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:17.855567    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="777b6c69-33b0-4454-9f22-8055e33104a9" path="/var/lib/kubelet/pods/777b6c69-33b0-4454-9f22-8055e33104a9/volumes"
Oct 30 22:34:19.370803 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:19.370757    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:20.028059 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:20.027996    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:20.146316 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:20.146259    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:20.247780 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:20.247750    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:24.749358 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.749298    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:24.897205 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.897168    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"]
Oct 30 22:34:24.897781 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.897757    5503 cpu_manager.go:414] "containerMap: removing container" podUID="777b6c69-33b0-4454-9f22-8055e33104a9" containerName="main"
Oct 30 22:34:24.897781 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.897781    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="777b6c69-33b0-4454-9f22-8055e33104a9" containerName="main"
Oct 30 22:34:24.898062 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.897928    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="777b6c69-33b0-4454-9f22-8055e33104a9" containerName="main"
Oct 30 22:34:24.902568 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.902542    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:24.908168 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.908138    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"openshift-service-ca.crt\""
Oct 30 22:34:24.909075 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.909047    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-kserve-self-signed-certs\""
Oct 30 22:34:24.909188 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.909103    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"scheduler-configmap-ref-test-epp-sa-dockercfg-55fpq\""
Oct 30 22:34:24.909512 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.909253    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"default-dockercfg-6m677\""
Oct 30 22:34:24.910216 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.910197    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-15d6126c\"/\"kube-root-ca.crt\""
Oct 30 22:34:24.918450 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.918423    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"]
Oct 30 22:34:24.924360 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.924330    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qh4b\" (UniqueName: \"kubernetes.io/projected/1f4f9448-3bef-4037-86a6-383f8b821fb7-kube-api-access-4qh4b\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d\" (UID: \"1f4f9448-3bef-4037-86a6-383f8b821fb7\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:24.924515 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.924388    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f4f9448-3bef-4037-86a6-383f8b821fb7-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d\" (UID: \"1f4f9448-3bef-4037-86a6-383f8b821fb7\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:24.949658 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:24.949623    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:25.025767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:25.025652    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qh4b\" (UniqueName: \"kubernetes.io/projected/1f4f9448-3bef-4037-86a6-383f8b821fb7-kube-api-access-4qh4b\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d\" (UID: \"1f4f9448-3bef-4037-86a6-383f8b821fb7\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:25.025767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:25.025716    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f4f9448-3bef-4037-86a6-383f8b821fb7-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d\" (UID: \"1f4f9448-3bef-4037-86a6-383f8b821fb7\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:25.028396 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:25.028368    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f4f9448-3bef-4037-86a6-383f8b821fb7-tls-certs\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d\" (UID: \"1f4f9448-3bef-4037-86a6-383f8b821fb7\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:25.039530 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:25.039496    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qh4b\" (UniqueName: \"kubernetes.io/projected/1f4f9448-3bef-4037-86a6-383f8b821fb7-kube-api-access-4qh4b\") pod \"scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d\" (UID: \"1f4f9448-3bef-4037-86a6-383f8b821fb7\") " pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:25.216495 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:25.216459    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:25.512136 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:25.512108    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"]
Oct 30 22:34:26.072204 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:26.072170    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" event={"ID":"1f4f9448-3bef-4037-86a6-383f8b821fb7","Type":"ContainerStarted","Data":"cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163"}
Oct 30 22:34:26.072731 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:26.072221    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" event={"ID":"1f4f9448-3bef-4037-86a6-383f8b821fb7","Type":"ContainerStarted","Data":"2afd62a0a74c02f42dbabf040a66a88e5a36243281886b7b2c468d1dcc064fd6"}
Oct 30 22:34:26.072731 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:26.072395    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:26.104692 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:26.095551    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" podStartSLOduration=2.095533433 podStartE2EDuration="2.095533433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:26.093409721 +0000 UTC m=+908.362491244" watchObservedRunningTime="2026-10-30 22:34:26.095533433 +0000 UTC m=+908.364614958"
Oct 30 22:34:27.251227 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:27.251139    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:30.711233 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:30.711164    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:33.295605 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:33.295555    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:33.404853 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:33.404822    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:34.764207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:34.764117    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:35.160837 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:35.160800    5503 generic.go:356] "Generic (PLEG): container finished" podID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerID="3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5" exitCode=0
Oct 30 22:34:35.161047 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:35.160875    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" event={"ID":"8b80c335-71f6-4e1c-acc5-fb6003572f92","Type":"ContainerDied","Data":"3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5"}
Oct 30 22:34:35.161254 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:35.161241    5503 scope.go:122] "RemoveContainer" containerID="3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5"
Oct 30 22:34:35.162343 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:35.162318    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:35.162397 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:35.162388    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:35.162431 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:35.162421    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:36.167837 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:36.167802    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" event={"ID":"8b80c335-71f6-4e1c-acc5-fb6003572f92","Type":"ContainerStarted","Data":"f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299"}
Oct 30 22:34:36.715777 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:36.715739    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:34:36.716066 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:36.716031    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-rb29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:34:36.716165 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:36.716106    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:34:37.707194 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:37.707154    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-rb29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:34:37.709945 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:37.709908    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:34:38.720050 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:38.720017    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-rb29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:34:38.720490 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:38.720071    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:34:38.839023 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:38.838982    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:34:39.717693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:39.716919    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-rb29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:34:39.717693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:39.716986    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:34:40.718694 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:40.718120    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-rb29g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body=
Oct 30 22:34:40.718694 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:40.718188    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused"
Oct 30 22:34:41.708424 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:41.708388    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:34:41.709364 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:41.709344    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:34:45.390117 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:34:45.390082    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:34:51.503648 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.503609    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"]
Oct 30 22:34:51.506719 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.506698    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.524781 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.524737    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"]
Oct 30 22:34:51.527485 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.527451    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"]
Oct 30 22:34:51.532772 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.532746    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.551567 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.551534    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"]
Oct 30 22:34:51.582220 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582186    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.582220 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582226    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.582472 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582244    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.582472 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582265    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.582472 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582329    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pz49\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-kube-api-access-5pz49\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.582590 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582495    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.582590 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582533    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.582590 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582559    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21edf88e-ee91-473c-830d-274ce2f3f1d5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.582712 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582591    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prwws\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-kube-api-access-prwws\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.582712 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582635    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.582712 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582681    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa22b625-db5f-47bf-8975-42293092dd4a-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.582817 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582774    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.582817 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582803    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa22b625-db5f-47bf-8975-42293092dd4a-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.582911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582898    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.582946 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582924    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.582946 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582940    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.583023 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.582957    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.583023 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.583005    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.684228 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684178    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.684228 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684230    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.684454 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684247    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.684454 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684369    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.684454 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684420    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.684584 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684454    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.684584 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684479    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.684584 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684508    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.684584 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684540    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.684584 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684565    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pz49\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-kube-api-access-5pz49\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.684863 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684608    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.684863 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684650    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.684863 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684697    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21edf88e-ee91-473c-830d-274ce2f3f1d5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.684863 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684718    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.684863 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684730    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prwws\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-kube-api-access-prwws\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.684863 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684809    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.685174 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684913    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.685174 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684985    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.685174 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.684996    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.685330 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.685180    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.685330 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.685230    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa22b625-db5f-47bf-8975-42293092dd4a-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.685330 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.685237    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.685330 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.685307    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.685330 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.685320    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.685549 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.685342    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa22b625-db5f-47bf-8975-42293092dd4a-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.685718 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.685694    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21edf88e-ee91-473c-830d-274ce2f3f1d5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.685813 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.685733    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.686083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.686055    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa22b625-db5f-47bf-8975-42293092dd4a-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.686889 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.686869    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.687234 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.687217    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.688150 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.688124    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa22b625-db5f-47bf-8975-42293092dd4a-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.688232 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.688201    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.697683 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.697640    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.698266 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.698243    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.698421 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.698402    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prwws\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-kube-api-access-prwws\") pod \"openshift-ai-inference-openshift-default-86b669b864-5k845\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.702587 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.702565    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pz49\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-kube-api-access-5pz49\") pod \"openshift-ai-inference-openshift-default-86b669b864-ks2t7\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:51.821225 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.821139    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:51.846825 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:51.846794    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:52.105393 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.105366    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"]
Oct 30 22:34:52.107373 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:34:52.107338    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21edf88e_ee91_473c_830d_274ce2f3f1d5.slice/crio-4bbbbfdd800401d0834bb54afd7050fef8c9d80ee9291a792069479de0b1c772 WatchSource:0}: Error finding container 4bbbbfdd800401d0834bb54afd7050fef8c9d80ee9291a792069479de0b1c772: Status 404 returned error can't find the container with id 4bbbbfdd800401d0834bb54afd7050fef8c9d80ee9291a792069479de0b1c772
Oct 30 22:34:52.109585 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.109543    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:52.109686 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.109608    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:52.109686 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.109635    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:52.143324 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.143303    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"]
Oct 30 22:34:52.146223 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:34:52.146196    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa22b625_db5f_47bf_8975_42293092dd4a.slice/crio-16411c9d372085a813851a6384deb452754289561d355e717481fd83e67eadc2 WatchSource:0}: Error finding container 16411c9d372085a813851a6384deb452754289561d355e717481fd83e67eadc2: Status 404 returned error can't find the container with id 16411c9d372085a813851a6384deb452754289561d355e717481fd83e67eadc2
Oct 30 22:34:52.148492 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.148464    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:52.148567 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.148521    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:52.148567 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.148554    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:52.254576 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.253725    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" event={"ID":"aa22b625-db5f-47bf-8975-42293092dd4a","Type":"ContainerStarted","Data":"9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b"}
Oct 30 22:34:52.254576 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.253789    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" event={"ID":"aa22b625-db5f-47bf-8975-42293092dd4a","Type":"ContainerStarted","Data":"16411c9d372085a813851a6384deb452754289561d355e717481fd83e67eadc2"}
Oct 30 22:34:52.257883 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.257208    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" event={"ID":"21edf88e-ee91-473c-830d-274ce2f3f1d5","Type":"ContainerStarted","Data":"c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997"}
Oct 30 22:34:52.257883 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.257268    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" event={"ID":"21edf88e-ee91-473c-830d-274ce2f3f1d5","Type":"ContainerStarted","Data":"4bbbbfdd800401d0834bb54afd7050fef8c9d80ee9291a792069479de0b1c772"}
Oct 30 22:34:52.312499 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.312426    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" podStartSLOduration=1.3124060659999999 podStartE2EDuration="1.312406066s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:52.311317101 +0000 UTC m=+934.580398623" watchObservedRunningTime="2026-10-30 22:34:52.312406066 +0000 UTC m=+934.581487590"
Oct 30 22:34:52.314460 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.313295    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" podStartSLOduration=1.313285034 podStartE2EDuration="1.313285034s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:52.286635539 +0000 UTC m=+934.555717063" watchObservedRunningTime="2026-10-30 22:34:52.313285034 +0000 UTC m=+934.582366559"
Oct 30 22:34:52.823812 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.823770    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:52.847678 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:52.847613    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:53.824158 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:53.824122    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5k845 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:34:53.824789 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:53.824748    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:34:53.848551 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:53.848515    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-ks2t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 22:34:53.848859 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:53.848832    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 22:34:54.824235 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:54.824198    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5k845 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:34:54.824897 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:54.824855    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:34:54.847434 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:54.847399    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-ks2t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 22:34:54.847616 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:54.847454    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 22:34:55.822111 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:55.822071    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5k845 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:34:55.822430 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:55.822403    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:34:55.848231 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:55.848189    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-ks2t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 22:34:55.848637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:55.848247    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 22:34:56.826694 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:56.822082    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5k845 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:34:56.826694 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:56.822153    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:34:56.847384 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:56.847343    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-ks2t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 22:34:56.847582 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:56.847412    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 22:34:57.078443 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:57.078359    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:34:57.823083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:57.823045    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5k845 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Oct 30 22:34:57.823367 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:57.823338    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Oct 30 22:34:57.851275 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:57.851233    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-ks2t7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body=
Oct 30 22:34:57.851450 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:57.851294    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused"
Oct 30 22:34:58.484289 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:58.484250    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"]
Oct 30 22:34:58.532741 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:58.532705    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:58.648375 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:58.648349    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:34:58.839307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:58.839228    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:58.839532 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:58.839515    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:58.840320 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:58.840304    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 22:34:58.852421 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:58.852396    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:58.852697 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:58.852657    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:34:58.853325 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:34:58.853306    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:35:00.061001 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:00.060962    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:00.168731 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:00.168696    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:04.580696 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:04.580639    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:07.078797 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:07.078758    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" podUID="1f4f9448-3bef-4037-86a6-383f8b821fb7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:35:09.445219 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:09.445180    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:09.877951 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:09.877919    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:10.015944 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:10.015911    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:13.507532 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:13.507492    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" podUID="1f4f9448-3bef-4037-86a6-383f8b821fb7" containerName="main" containerID="cri-o://cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163" gracePeriod=45
Oct 30 22:35:15.393881 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:35:15.393816    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:35:15.502554 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.502518    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:15.688111 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.688029    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"]
Oct 30 22:35:15.691841 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.691821    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:35:15.706646 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.706618    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"]
Oct 30 22:35:15.708355 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.708332    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c\"/\"kube-root-ca.crt\""
Oct 30 22:35:15.708648 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.708334    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c\"/\"openshift-service-ca.crt\""
Oct 30 22:35:15.708815 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.708382    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c\"/\"test-cpu-accelerator-preset-ser-afc820a4-epp-sa-dockercfg-tvzsg\""
Oct 30 22:35:15.708815 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.708414    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c\"/\"default-dockercfg-4t64m\""
Oct 30 22:35:15.709431 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.709411    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c\"/\"test-cb25a14c9ba423627edeb5388343bd353-kserve-self-signed-certs\""
Oct 30 22:35:15.734765 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.734734    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:15.741945 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.741915    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-tls-certs\") pod \"test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg\" (UID: \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\") " pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:35:15.742083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.741991    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdz4s\" (UniqueName: \"kubernetes.io/projected/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-kube-api-access-rdz4s\") pod \"test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg\" (UID: \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\") " pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:35:15.843034 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.842991    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdz4s\" (UniqueName: \"kubernetes.io/projected/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-kube-api-access-rdz4s\") pod \"test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg\" (UID: \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\") " pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:35:15.843248 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.843079    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-tls-certs\") pod \"test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg\" (UID: \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\") " pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:35:15.845769 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.845740    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-tls-certs\") pod \"test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg\" (UID: \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\") " pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:35:15.857650 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:15.857626    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdz4s\" (UniqueName: \"kubernetes.io/projected/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-kube-api-access-rdz4s\") pod \"test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg\" (UID: \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\") " pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:35:16.005979 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:16.005893    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:35:16.282127 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:16.282097    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"]
Oct 30 22:35:16.284129 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:35:16.284100    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde4f7c1_a6e1_484c_a254_2af2487bf4f9.slice/crio-3565e5baf4e32b41db9fa64adbedd635a1533076a4caa6d328ec62440ddea0bf WatchSource:0}: Error finding container 3565e5baf4e32b41db9fa64adbedd635a1533076a4caa6d328ec62440ddea0bf: Status 404 returned error can't find the container with id 3565e5baf4e32b41db9fa64adbedd635a1533076a4caa6d328ec62440ddea0bf
Oct 30 22:35:16.394634 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:16.394602    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" event={"ID":"fde4f7c1-a6e1-484c-a254-2af2487bf4f9","Type":"ContainerStarted","Data":"207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5"}
Oct 30 22:35:16.395031 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:16.394659    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" event={"ID":"fde4f7c1-a6e1-484c-a254-2af2487bf4f9","Type":"ContainerStarted","Data":"3565e5baf4e32b41db9fa64adbedd635a1533076a4caa6d328ec62440ddea0bf"}
Oct 30 22:35:16.395031 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:16.394725    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:35:17.078227 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:17.078187    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" podUID="1f4f9448-3bef-4037-86a6-383f8b821fb7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:35:17.106506 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:17.106471    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:17.259281 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:17.259249    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:19.429996 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:19.429965    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:27.078488 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:27.078445    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" podUID="1f4f9448-3bef-4037-86a6-383f8b821fb7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:35:27.078941 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:27.078569    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:35:27.109433 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:27.109379    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" podStartSLOduration=12.109365205 podStartE2EDuration="12.109365205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:35:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:35:16.416846581 +0000 UTC m=+958.685928106" watchObservedRunningTime="2026-10-30 22:35:27.109365205 +0000 UTC m=+969.378446728"
Oct 30 22:35:27.450038 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:27.450002    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:35:37.078913 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:37.078876    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" podUID="1f4f9448-3bef-4037-86a6-383f8b821fb7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:35:43.754002 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:43.753974    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:35:43.826154 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:43.826070    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f4f9448-3bef-4037-86a6-383f8b821fb7-kube-api-access-4qh4b\" (UniqueName: \"kubernetes.io/projected/1f4f9448-3bef-4037-86a6-383f8b821fb7-kube-api-access-4qh4b\") pod \"1f4f9448-3bef-4037-86a6-383f8b821fb7\" (UID: \"1f4f9448-3bef-4037-86a6-383f8b821fb7\") "
Oct 30 22:35:43.826154 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:43.826126    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f4f9448-3bef-4037-86a6-383f8b821fb7-tls-certs\" (UniqueName: \"kubernetes.io/secret/1f4f9448-3bef-4037-86a6-383f8b821fb7-tls-certs\") pod \"1f4f9448-3bef-4037-86a6-383f8b821fb7\" (UID: \"1f4f9448-3bef-4037-86a6-383f8b821fb7\") "
Oct 30 22:35:43.828314 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:43.828284    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f4f9448-3bef-4037-86a6-383f8b821fb7-tls-certs" pod "1f4f9448-3bef-4037-86a6-383f8b821fb7" (UID: "1f4f9448-3bef-4037-86a6-383f8b821fb7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:35:43.828314 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:43.828287    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4f9448-3bef-4037-86a6-383f8b821fb7-kube-api-access-4qh4b" pod "1f4f9448-3bef-4037-86a6-383f8b821fb7" (UID: "1f4f9448-3bef-4037-86a6-383f8b821fb7"). InnerVolumeSpecName "kube-api-access-4qh4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:35:43.927263 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:43.927220    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qh4b\" (UniqueName: \"kubernetes.io/projected/1f4f9448-3bef-4037-86a6-383f8b821fb7-kube-api-access-4qh4b\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:43.927263 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:43.927262    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f4f9448-3bef-4037-86a6-383f8b821fb7-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:35:44.537391 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:44.537354    5503 generic.go:356] "Generic (PLEG): container finished" podID="1f4f9448-3bef-4037-86a6-383f8b821fb7" containerID="cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163" exitCode=0
Oct 30 22:35:44.537602 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:44.537413    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"
Oct 30 22:35:44.537602 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:44.537436    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" event={"ID":"1f4f9448-3bef-4037-86a6-383f8b821fb7","Type":"ContainerDied","Data":"cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163"}
Oct 30 22:35:44.537602 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:44.537500    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d" event={"ID":"1f4f9448-3bef-4037-86a6-383f8b821fb7","Type":"ContainerDied","Data":"2afd62a0a74c02f42dbabf040a66a88e5a36243281886b7b2c468d1dcc064fd6"}
Oct 30 22:35:44.537602 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:44.537528    5503 scope.go:122] "RemoveContainer" containerID="cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163"
Oct 30 22:35:44.546205 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:44.546184    5503 scope.go:122] "RemoveContainer" containerID="cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163"
Oct 30 22:35:44.546492 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:35:44.546469    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163\": container with ID starting with cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163 not found: ID does not exist" containerID="cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163"
Oct 30 22:35:44.546541 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:44.546499    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163"} err="failed to get container status \"cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163\": rpc error: code = NotFound desc = could not find container \"cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163\": container with ID starting with cf176f2b32557d2e2dcbea2416f787659cb60cc382ea50395f91bac50d480163 not found: ID does not exist"
Oct 30 22:35:44.556896 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:44.556864    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"]
Oct 30 22:35:44.561228 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:44.561203    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-15d6126c/scheduler-configmap-ref-test-kserve-router-scheduler-7599bxsw2d"]
Oct 30 22:35:45.389992 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:35:45.389956    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:35:45.855605 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:45.855555    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f4f9448-3bef-4037-86a6-383f8b821fb7" path="/var/lib/kubelet/pods/1f4f9448-3bef-4037-86a6-383f8b821fb7/volumes"
Oct 30 22:35:47.402244 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:35:47.402214    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:36:00.954444 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:00.954409    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:04.631174 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:04.631138    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:15.390636 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:36:15.390599    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:36:16.604482 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:16.604447    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:19.500551 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:19.500517    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:39.526366 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:39.526327    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"]
Oct 30 22:36:39.563596 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:39.563550    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:39.682860 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:39.682829    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:36:45.390521 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:36:45.390483    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:36:47.402901 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:47.402861    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" podUID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:36:54.547234 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:54.547198    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" podUID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" containerName="main" containerID="cri-o://207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5" gracePeriod=45
Oct 30 22:36:57.402638 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:36:57.402596    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" podUID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:37:07.402280 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:07.402242    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" podUID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:37:07.402760 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:07.402378    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:37:14.498524 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:14.498487    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:14.699954 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:14.699925    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:15.389656 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:37:15.389620    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:37:17.402769 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:17.402728    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" podUID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:37:19.555100 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:19.555062    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:24.798910 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:24.798880    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:37:24.876491 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:24.876460    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-kube-api-access-rdz4s\" (UniqueName: \"kubernetes.io/projected/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-kube-api-access-rdz4s\") pod \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\" (UID: \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\") "
Oct 30 22:37:24.876700 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:24.876527    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-tls-certs\" (UniqueName: \"kubernetes.io/secret/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-tls-certs\") pod \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\" (UID: \"fde4f7c1-a6e1-484c-a254-2af2487bf4f9\") "
Oct 30 22:37:24.878934 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:24.878901    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-tls-certs" pod "fde4f7c1-a6e1-484c-a254-2af2487bf4f9" (UID: "fde4f7c1-a6e1-484c-a254-2af2487bf4f9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:37:24.878934 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:24.878904    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-kube-api-access-rdz4s" pod "fde4f7c1-a6e1-484c-a254-2af2487bf4f9" (UID: "fde4f7c1-a6e1-484c-a254-2af2487bf4f9"). InnerVolumeSpecName "kube-api-access-rdz4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:37:24.977270 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:24.977233    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdz4s\" (UniqueName: \"kubernetes.io/projected/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-kube-api-access-rdz4s\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:24.977270 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:24.977269    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fde4f7c1-a6e1-484c-a254-2af2487bf4f9-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:37:25.053903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.053869    5503 generic.go:356] "Generic (PLEG): container finished" podID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" containerID="207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5" exitCode=0
Oct 30 22:37:25.054073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.053916    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"
Oct 30 22:37:25.054073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.053935    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" event={"ID":"fde4f7c1-a6e1-484c-a254-2af2487bf4f9","Type":"ContainerDied","Data":"207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5"}
Oct 30 22:37:25.054073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.053977    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg" event={"ID":"fde4f7c1-a6e1-484c-a254-2af2487bf4f9","Type":"ContainerDied","Data":"3565e5baf4e32b41db9fa64adbedd635a1533076a4caa6d328ec62440ddea0bf"}
Oct 30 22:37:25.054073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.053997    5503 scope.go:122] "RemoveContainer" containerID="207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5"
Oct 30 22:37:25.065758 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.065737    5503 scope.go:122] "RemoveContainer" containerID="207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5"
Oct 30 22:37:25.066071 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:37:25.066047    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5\": container with ID starting with 207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5 not found: ID does not exist" containerID="207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5"
Oct 30 22:37:25.066128 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.066080    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5"} err="failed to get container status \"207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5\": rpc error: code = NotFound desc = could not find container \"207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5\": container with ID starting with 207bb6d1ac367efef6841f867326605060f79cad9dbd5bebc95aee1e9aee19d5 not found: ID does not exist"
Oct 30 22:37:25.079140 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.079113    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"]
Oct 30 22:37:25.085347 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.085322    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-cpu-accelerator-preset-serves-inference-a4a93d2c/test-cpb25a14c9ba423627edeb5388343bd353-kserve-router-schehlwsg"]
Oct 30 22:37:25.855445 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:25.855411    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" path="/var/lib/kubelet/pods/fde4f7c1-a6e1-484c-a254-2af2487bf4f9/volumes"
Oct 30 22:37:29.929858 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:29.929582    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:30.085923 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:30.085890    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:32.804954 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.804917    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:32.957767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.957734    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"]
Oct 30 22:37:32.958125 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.958109    5503 cpu_manager.go:414] "containerMap: removing container" podUID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" containerName="main"
Oct 30 22:37:32.958125 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.958124    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" containerName="main"
Oct 30 22:37:32.958270 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.958157    5503 cpu_manager.go:414] "containerMap: removing container" podUID="1f4f9448-3bef-4037-86a6-383f8b821fb7" containerName="main"
Oct 30 22:37:32.958270 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.958162    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f4f9448-3bef-4037-86a6-383f8b821fb7" containerName="main"
Oct 30 22:37:32.958270 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.958223    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f4f9448-3bef-4037-86a6-383f8b821fb7" containerName="main"
Oct 30 22:37:32.958270 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.958234    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="fde4f7c1-a6e1-484c-a254-2af2487bf4f9" containerName="main"
Oct 30 22:37:32.961352 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.961332    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:37:32.969394 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.969204    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"kube-root-ca.crt\""
Oct 30 22:37:32.974361 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.974332    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"]
Oct 30 22:37:32.974761 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.974726    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-epp-sa-dockercfg-6fc5m\""
Oct 30 22:37:32.977303 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:32.977278    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"fc-smoke-test-kserve-self-signed-certs\""
Oct 30 22:37:33.012355 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.012326    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"default-dockercfg-xcq9r\""
Oct 30 22:37:33.052871 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.052836    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c718022-345d-4b4a-aebc-1fff34158d6a-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd\" (UID: \"0c718022-345d-4b4a-aebc-1fff34158d6a\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:37:33.053049 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.053000    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgrnh\" (UniqueName: \"kubernetes.io/projected/0c718022-345d-4b4a-aebc-1fff34158d6a-kube-api-access-mgrnh\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd\" (UID: \"0c718022-345d-4b4a-aebc-1fff34158d6a\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:37:33.074005 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.073919    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-4f8df83c\"/\"openshift-service-ca.crt\""
Oct 30 22:37:33.076374 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.076347    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:33.154518 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.154484    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c718022-345d-4b4a-aebc-1fff34158d6a-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd\" (UID: \"0c718022-345d-4b4a-aebc-1fff34158d6a\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:37:33.154748 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.154578    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgrnh\" (UniqueName: \"kubernetes.io/projected/0c718022-345d-4b4a-aebc-1fff34158d6a-kube-api-access-mgrnh\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd\" (UID: \"0c718022-345d-4b4a-aebc-1fff34158d6a\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:37:33.157309 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.157284    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c718022-345d-4b4a-aebc-1fff34158d6a-tls-certs\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd\" (UID: \"0c718022-345d-4b4a-aebc-1fff34158d6a\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:37:33.187541 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.187504    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgrnh\" (UniqueName: \"kubernetes.io/projected/0c718022-345d-4b4a-aebc-1fff34158d6a-kube-api-access-mgrnh\") pod \"fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd\" (UID: \"0c718022-345d-4b4a-aebc-1fff34158d6a\") " pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:37:33.276812 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.276774    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:37:33.567942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:33.567914    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"]
Oct 30 22:37:33.570280 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:37:33.570253    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c718022_345d_4b4a_aebc_1fff34158d6a.slice/crio-702e3ac42e528d954974d0872c11f925af0e9d2c8caddd1ac662a05ce24cc70e WatchSource:0}: Error finding container 702e3ac42e528d954974d0872c11f925af0e9d2c8caddd1ac662a05ce24cc70e: Status 404 returned error can't find the container with id 702e3ac42e528d954974d0872c11f925af0e9d2c8caddd1ac662a05ce24cc70e
Oct 30 22:37:34.132691 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:34.132377    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" event={"ID":"0c718022-345d-4b4a-aebc-1fff34158d6a","Type":"ContainerStarted","Data":"3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17"}
Oct 30 22:37:34.132691 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:34.132449    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" event={"ID":"0c718022-345d-4b4a-aebc-1fff34158d6a","Type":"ContainerStarted","Data":"702e3ac42e528d954974d0872c11f925af0e9d2c8caddd1ac662a05ce24cc70e"}
Oct 30 22:37:34.136700 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:34.133370    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:37:36.371565 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:36.371532    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:42.302848 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:42.302814    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:42.680633 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:42.680604    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:42.795745 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:37:42.795701    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:37:45.391951 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:37:45.391917    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:38:06.184794 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:06.184760    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:38:06.208234 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:06.208179    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" podStartSLOduration=34.208166243 podStartE2EDuration="34.208166243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:37:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:37:34.162418186 +0000 UTC m=+1096.431499712" watchObservedRunningTime="2026-10-30 22:38:06.208166243 +0000 UTC m=+1128.477247765"
Oct 30 22:38:07.523926 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:07.523893    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:13.588729 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:13.588659    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"]
Oct 30 22:38:13.613352 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:13.613321    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:13.775961 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:13.775922    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:14.564272 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:14.564236    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:15.390705 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:38:15.390651    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:38:16.184903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:16.184861    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:38:19.634319 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:19.634286    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:38:26.184565 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:26.184527    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:38:28.609197 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:28.609155    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerName="main" containerID="cri-o://3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17" gracePeriod=45
Oct 30 22:38:36.184959 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:36.184923    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:38:36.185388 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:36.185029    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:38:45.389989 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:38:45.389949    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:38:46.185293 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:46.185256    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:38:56.184591 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:56.184554    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:38:58.852585 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:58.852553    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:38:58.933882 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:58.933777    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0c718022-345d-4b4a-aebc-1fff34158d6a-tls-certs\" (UniqueName: \"kubernetes.io/secret/0c718022-345d-4b4a-aebc-1fff34158d6a-tls-certs\") pod \"0c718022-345d-4b4a-aebc-1fff34158d6a\" (UID: \"0c718022-345d-4b4a-aebc-1fff34158d6a\") "
Oct 30 22:38:58.933882 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:58.933869    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c718022-345d-4b4a-aebc-1fff34158d6a-kube-api-access-mgrnh\" (UniqueName: \"kubernetes.io/projected/0c718022-345d-4b4a-aebc-1fff34158d6a-kube-api-access-mgrnh\") pod \"0c718022-345d-4b4a-aebc-1fff34158d6a\" (UID: \"0c718022-345d-4b4a-aebc-1fff34158d6a\") "
Oct 30 22:38:58.936267 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:58.936228    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c718022-345d-4b4a-aebc-1fff34158d6a-tls-certs" pod "0c718022-345d-4b4a-aebc-1fff34158d6a" (UID: "0c718022-345d-4b4a-aebc-1fff34158d6a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:38:58.936393 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:58.936290    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c718022-345d-4b4a-aebc-1fff34158d6a-kube-api-access-mgrnh" pod "0c718022-345d-4b4a-aebc-1fff34158d6a" (UID: "0c718022-345d-4b4a-aebc-1fff34158d6a"). InnerVolumeSpecName "kube-api-access-mgrnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:38:59.035217 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.035183    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c718022-345d-4b4a-aebc-1fff34158d6a-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:59.035217 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.035213    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgrnh\" (UniqueName: \"kubernetes.io/projected/0c718022-345d-4b4a-aebc-1fff34158d6a-kube-api-access-mgrnh\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:38:59.648826 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.648786    5503 generic.go:356] "Generic (PLEG): container finished" podID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerID="3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17" exitCode=0
Oct 30 22:38:59.649003 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.648847    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" event={"ID":"0c718022-345d-4b4a-aebc-1fff34158d6a","Type":"ContainerDied","Data":"3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17"}
Oct 30 22:38:59.649003 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.648857    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"
Oct 30 22:38:59.649003 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.648928    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd" event={"ID":"0c718022-345d-4b4a-aebc-1fff34158d6a","Type":"ContainerDied","Data":"702e3ac42e528d954974d0872c11f925af0e9d2c8caddd1ac662a05ce24cc70e"}
Oct 30 22:38:59.649003 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.648951    5503 scope.go:122] "RemoveContainer" containerID="3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17"
Oct 30 22:38:59.661502 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.661478    5503 scope.go:122] "RemoveContainer" containerID="3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17"
Oct 30 22:38:59.661831 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:38:59.661808    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17\": container with ID starting with 3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17 not found: ID does not exist" containerID="3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17"
Oct 30 22:38:59.661921 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.661840    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17"} err="failed to get container status \"3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17\": rpc error: code = NotFound desc = could not find container \"3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17\": container with ID starting with 3a8ee850468c20838adfde5ff28743862c49b6a3cae0cebee25f674c561e6c17 not found: ID does not exist"
Oct 30 22:38:59.679184 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.679147    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"]
Oct 30 22:38:59.681369 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.681337    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-4f8df83c/fc-smoke-test-kserve-router-scheduler-77fbbb774b-dwwxd"]
Oct 30 22:38:59.855047 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:38:59.855016    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" path="/var/lib/kubelet/pods/0c718022-345d-4b4a-aebc-1fff34158d6a/volumes"
Oct 30 22:39:03.737583 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:03.737544    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:04.958933 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:04.958869    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:07.647308 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.647272    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:07.873648 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.873612    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"]
Oct 30 22:39:07.874092 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.874074    5503 cpu_manager.go:414] "containerMap: removing container" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerName="main"
Oct 30 22:39:07.874189 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.874094    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerName="main"
Oct 30 22:39:07.874264 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.874249    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c718022-345d-4b4a-aebc-1fff34158d6a" containerName="main"
Oct 30 22:39:07.877393 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.877369    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:07.882467 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.882442    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"openshift-service-ca.crt\""
Oct 30 22:39:07.883453 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.883406    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"default-dockercfg-44642\""
Oct 30 22:39:07.883453 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.883425    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-epp-sa-dockercfg-9hgjg\""
Oct 30 22:39:07.888620 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.888598    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"fc-concurrency-test-kserve-self-signed-certs\""
Oct 30 22:39:07.891809 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.891772    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"]
Oct 30 22:39:07.949442 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.949353    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:07.983227 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:07.983198    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-flow-control-smoke-1e98f46d\"/\"kube-root-ca.crt\""
Oct 30 22:39:08.029714 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:08.029659    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpxvk\" (UniqueName: \"kubernetes.io/projected/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-kube-api-access-dpxvk\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5\" (UID: \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:08.029897 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:08.029732    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5\" (UID: \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:08.130527 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:08.130490    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpxvk\" (UniqueName: \"kubernetes.io/projected/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-kube-api-access-dpxvk\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5\" (UID: \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:08.130738 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:08.130540    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5\" (UID: \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:08.133274 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:08.133244    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-tls-certs\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5\" (UID: \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:08.143748 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:08.143721    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpxvk\" (UniqueName: \"kubernetes.io/projected/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-kube-api-access-dpxvk\") pod \"fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5\" (UID: \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\") " pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:08.192750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:08.192705    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:08.487940 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:08.481936    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"]
Oct 30 22:39:08.737691 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:08.734162    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" event={"ID":"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1","Type":"ContainerStarted","Data":"14c664408c8602698e4d4d482c91ae2e9d55b370a09bd9479dbd64895a36555b"}
Oct 30 22:39:09.763920 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:09.763881    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" event={"ID":"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1","Type":"ContainerStarted","Data":"8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d"}
Oct 30 22:39:09.765372 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:09.765348    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:10.427587 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:10.427552    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:14.590420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:14.590348    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:15.390023 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:39:15.389989    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:39:15.643914 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:15.643822    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:16.593164 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:16.593130    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:16.767170 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:16.767138    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:18.502612 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:18.502579    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:18.678639 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:18.678607    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:19.714280 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:19.714241    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:25.268880 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:25.268844    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:31.441026 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:31.440963    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:31.551165 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:31.551125    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:33.564224 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.564187    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:33.726017 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.725950    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" podStartSLOduration=26.725931405 podStartE2EDuration="26.725931405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:39:09.813742135 +0000 UTC m=+1192.082823658" watchObservedRunningTime="2026-10-30 22:39:33.725931405 +0000 UTC m=+1215.995012928"
Oct 30 22:39:33.728089 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.728064    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"]
Oct 30 22:39:33.732592 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.732567    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:39:33.745258 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.745230    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Oct 30 22:39:33.746086 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.746066    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"]
Oct 30 22:39:33.746559 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.746534    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Oct 30 22:39:33.746689 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.746613    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Oct 30 22:39:33.746689 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.746611    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-5lf77\""
Oct 30 22:39:33.746689 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.746641    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-epp-sa-dockercfg-hb45s\""
Oct 30 22:39:33.781346 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.781315    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:33.815408 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.815294    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11a066dd-c225-43be-a929-4e9db7793a4c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z\" (UID: \"11a066dd-c225-43be-a929-4e9db7793a4c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:39:33.815588 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.815463    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdkpz\" (UniqueName: \"kubernetes.io/projected/11a066dd-c225-43be-a929-4e9db7793a4c-kube-api-access-cdkpz\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z\" (UID: \"11a066dd-c225-43be-a929-4e9db7793a4c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:39:33.916599 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.916546    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdkpz\" (UniqueName: \"kubernetes.io/projected/11a066dd-c225-43be-a929-4e9db7793a4c-kube-api-access-cdkpz\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z\" (UID: \"11a066dd-c225-43be-a929-4e9db7793a4c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:39:33.916895 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.916626    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11a066dd-c225-43be-a929-4e9db7793a4c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z\" (UID: \"11a066dd-c225-43be-a929-4e9db7793a4c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:39:33.926845 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.925747    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11a066dd-c225-43be-a929-4e9db7793a4c-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z\" (UID: \"11a066dd-c225-43be-a929-4e9db7793a4c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:39:33.942802 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:33.942769    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdkpz\" (UniqueName: \"kubernetes.io/projected/11a066dd-c225-43be-a929-4e9db7793a4c-kube-api-access-cdkpz\") pod \"scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z\" (UID: \"11a066dd-c225-43be-a929-4e9db7793a4c\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:39:34.048725 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:34.048652    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:39:34.346447 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:34.346422    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"]
Oct 30 22:39:34.348865 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:39:34.348818    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11a066dd_c225_43be_a929_4e9db7793a4c.slice/crio-d30c4ded408d5f13745c414dce89f7a4d9d75c4575c2f77f420df58805423cfb WatchSource:0}: Error finding container d30c4ded408d5f13745c414dce89f7a4d9d75c4575c2f77f420df58805423cfb: Status 404 returned error can't find the container with id d30c4ded408d5f13745c414dce89f7a4d9d75c4575c2f77f420df58805423cfb
Oct 30 22:39:34.953550 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:34.953509    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" event={"ID":"11a066dd-c225-43be-a929-4e9db7793a4c","Type":"ContainerStarted","Data":"58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108"}
Oct 30 22:39:34.956073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:34.953574    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" event={"ID":"11a066dd-c225-43be-a929-4e9db7793a4c","Type":"ContainerStarted","Data":"d30c4ded408d5f13745c414dce89f7a4d9d75c4575c2f77f420df58805423cfb"}
Oct 30 22:39:34.956073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:34.953600    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:39:37.268482 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:37.268445    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:41.801840 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:41.801805    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:39:41.823337 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:41.823282    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" podStartSLOduration=8.823269548 podStartE2EDuration="8.823269548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:39:34.978141282 +0000 UTC m=+1217.247222816" watchObservedRunningTime="2026-10-30 22:39:41.823269548 +0000 UTC m=+1224.092351071"
Oct 30 22:39:45.390960 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:39:45.390922    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:39:48.320781 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:48.320745    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"]
Oct 30 22:39:48.322195 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:48.322169    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:48.434479 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:48.434389    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:39:51.801061 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:39:51.801023    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:40:01.801094 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:01.801056    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:40:03.341329 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:03.341290    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerName="main" containerID="cri-o://8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d" gracePeriod=45
Oct 30 22:40:05.971000 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:05.970968    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:40:07.839715 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:07.839658    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"]
Oct 30 22:40:07.874152 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:07.874122    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:08.029101 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:08.029068    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:11.801392 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:11.801358    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:40:11.801838 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:11.801451    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:40:14.638192 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:14.638153    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:15.390353 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:40:15.390316    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:40:15.971171 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:15.971136    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" podUID="11a066dd-c225-43be-a929-4e9db7793a4c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:40:19.707832 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:19.707801    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:21.801928 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:21.801892    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:40:22.861658 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:22.861613    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" podUID="11a066dd-c225-43be-a929-4e9db7793a4c" containerName="main" containerID="cri-o://58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108" gracePeriod=45
Oct 30 22:40:25.971507 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:25.971470    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" podUID="11a066dd-c225-43be-a929-4e9db7793a4c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:40:31.801196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:31.801093    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:40:33.586752 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:33.586726    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:40:33.620818 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:33.620735    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-kube-api-access-dpxvk\" (UniqueName: \"kubernetes.io/projected/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-kube-api-access-dpxvk\") pod \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\" (UID: \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\") "
Oct 30 22:40:33.620997 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:33.620824    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-tls-certs\" (UniqueName: \"kubernetes.io/secret/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-tls-certs\") pod \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\" (UID: \"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1\") "
Oct 30 22:40:33.623133 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:33.623105    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-tls-certs" pod "b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" (UID: "b34668f8-dc1b-45c3-86f6-f9fdc1b564c1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:33.623234 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:33.623178    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-kube-api-access-dpxvk" pod "b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" (UID: "b34668f8-dc1b-45c3-86f6-f9fdc1b564c1"). InnerVolumeSpecName "kube-api-access-dpxvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:33.721430 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:33.721386    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:33.721430 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:33.721423    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpxvk\" (UniqueName: \"kubernetes.io/projected/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1-kube-api-access-dpxvk\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:34.271222 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:34.271185    5503 generic.go:356] "Generic (PLEG): container finished" podID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerID="8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d" exitCode=0
Oct 30 22:40:34.271435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:34.271230    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"
Oct 30 22:40:34.271435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:34.271273    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" event={"ID":"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1","Type":"ContainerDied","Data":"8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d"}
Oct 30 22:40:34.271435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:34.271333    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5" event={"ID":"b34668f8-dc1b-45c3-86f6-f9fdc1b564c1","Type":"ContainerDied","Data":"14c664408c8602698e4d4d482c91ae2e9d55b370a09bd9479dbd64895a36555b"}
Oct 30 22:40:34.271435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:34.271361    5503 scope.go:122] "RemoveContainer" containerID="8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d"
Oct 30 22:40:34.280302 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:34.280283    5503 scope.go:122] "RemoveContainer" containerID="8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d"
Oct 30 22:40:34.280572 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:40:34.280553    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d\": container with ID starting with 8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d not found: ID does not exist" containerID="8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d"
Oct 30 22:40:34.280614 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:34.280580    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d"} err="failed to get container status \"8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d\": rpc error: code = NotFound desc = could not find container \"8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d\": container with ID starting with 8f4bc98640e788bcf273eee70a9203001498aeefc26e47de2595df8e114c144d not found: ID does not exist"
Oct 30 22:40:34.292960 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:34.292931    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"]
Oct 30 22:40:34.296915 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:34.296892    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-flow-control-smoke-1e98f46d/fc-concurrency-test-kserve-router-scheduler-7db7c65fd7-qwpb5"]
Oct 30 22:40:35.854574 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:35.854541    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" path="/var/lib/kubelet/pods/b34668f8-dc1b-45c3-86f6-f9fdc1b564c1/volumes"
Oct 30 22:40:35.971122 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:35.971081    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" podUID="11a066dd-c225-43be-a929-4e9db7793a4c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:40:35.971266 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:35.971176    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:40:38.585279 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:38.585192    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:38.737385 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:38.737343    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:38.874817 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:38.874786    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:39.183613 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:39.183533    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:39.284550 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:39.284519    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:42.531825 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:42.531771    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:45.131525 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:45.131486    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:45.390214 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:40:45.390127    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:40:45.970904 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:45.970867    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" podUID="11a066dd-c225-43be-a929-4e9db7793a4c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:40:46.676004 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:46.675976    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:48.041424 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:48.041393    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:49.018826 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.018802    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-2541cd65_scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z_11a066dd-c225-43be-a929-4e9db7793a4c/main/0.log"
Oct 30 22:40:49.021196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.021168    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:40:49.059835 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.059750    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/11a066dd-c225-43be-a929-4e9db7793a4c-tls-certs\" (UniqueName: \"kubernetes.io/secret/11a066dd-c225-43be-a929-4e9db7793a4c-tls-certs\") pod \"11a066dd-c225-43be-a929-4e9db7793a4c\" (UID: \"11a066dd-c225-43be-a929-4e9db7793a4c\") "
Oct 30 22:40:49.060221 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.059935    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11a066dd-c225-43be-a929-4e9db7793a4c-kube-api-access-cdkpz\" (UniqueName: \"kubernetes.io/projected/11a066dd-c225-43be-a929-4e9db7793a4c-kube-api-access-cdkpz\") pod \"11a066dd-c225-43be-a929-4e9db7793a4c\" (UID: \"11a066dd-c225-43be-a929-4e9db7793a4c\") "
Oct 30 22:40:49.062088 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.062054    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11a066dd-c225-43be-a929-4e9db7793a4c-tls-certs" pod "11a066dd-c225-43be-a929-4e9db7793a4c" (UID: "11a066dd-c225-43be-a929-4e9db7793a4c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:49.062199 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.062152    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11a066dd-c225-43be-a929-4e9db7793a4c-kube-api-access-cdkpz" pod "11a066dd-c225-43be-a929-4e9db7793a4c" (UID: "11a066dd-c225-43be-a929-4e9db7793a4c"). InnerVolumeSpecName "kube-api-access-cdkpz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:49.161088 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.161045    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdkpz\" (UniqueName: \"kubernetes.io/projected/11a066dd-c225-43be-a929-4e9db7793a4c-kube-api-access-cdkpz\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:49.161088 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.161077    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/11a066dd-c225-43be-a929-4e9db7793a4c-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:49.356113 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.356087    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-2541cd65_scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z_11a066dd-c225-43be-a929-4e9db7793a4c/main/0.log"
Oct 30 22:40:49.356289 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.356130    5503 generic.go:356] "Generic (PLEG): container finished" podID="11a066dd-c225-43be-a929-4e9db7793a4c" containerID="58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108" exitCode=1
Oct 30 22:40:49.356289 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.356181    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"
Oct 30 22:40:49.356289 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.356227    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" event={"ID":"11a066dd-c225-43be-a929-4e9db7793a4c","Type":"ContainerDied","Data":"58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108"}
Oct 30 22:40:49.356289 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.356282    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z" event={"ID":"11a066dd-c225-43be-a929-4e9db7793a4c","Type":"ContainerDied","Data":"d30c4ded408d5f13745c414dce89f7a4d9d75c4575c2f77f420df58805423cfb"}
Oct 30 22:40:49.356417 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.356304    5503 scope.go:122] "RemoveContainer" containerID="58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108"
Oct 30 22:40:49.365634 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.365613    5503 scope.go:122] "RemoveContainer" containerID="58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108"
Oct 30 22:40:49.365925 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:40:49.365905    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108\": container with ID starting with 58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108 not found: ID does not exist" containerID="58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108"
Oct 30 22:40:49.365997 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.365938    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108"} err="failed to get container status \"58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108\": rpc error: code = NotFound desc = could not find container \"58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108\": container with ID starting with 58ef2cdfe70b4f47a037bb41d8e3ca1c925a0d88ec67c580aef4a3372b2a1108 not found: ID does not exist"
Oct 30 22:40:49.380210 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.380171    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"]
Oct 30 22:40:49.386403 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.386374    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-router-scheduler-84865b67f27z"]
Oct 30 22:40:49.854796 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:49.854764    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11a066dd-c225-43be-a929-4e9db7793a4c" path="/var/lib/kubelet/pods/11a066dd-c225-43be-a929-4e9db7793a4c/volumes"
Oct 30 22:40:50.845763 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:50.845729    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:50.948519 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:50.948486    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:53.109333 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:53.109299    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:53.224173 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:53.224140    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:58.189007 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:58.188949    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:58.556869 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:58.556744    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:59.191482 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:59.191449    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:40:59.933640 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:40:59.933608    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:00.120555 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.120518    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"]
Oct 30 22:41:00.121132 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.121111    5503 cpu_manager.go:414] "containerMap: removing container" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerName="main"
Oct 30 22:41:00.121225 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.121135    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerName="main"
Oct 30 22:41:00.121225 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.121185    5503 cpu_manager.go:414] "containerMap: removing container" podUID="11a066dd-c225-43be-a929-4e9db7793a4c" containerName="main"
Oct 30 22:41:00.121225 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.121193    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11a066dd-c225-43be-a929-4e9db7793a4c" containerName="main"
Oct 30 22:41:00.121327 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.121280    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="b34668f8-dc1b-45c3-86f6-f9fdc1b564c1" containerName="main"
Oct 30 22:41:00.121327 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.121303    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="11a066dd-c225-43be-a929-4e9db7793a4c" containerName="main"
Oct 30 22:41:00.125991 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.125973    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.132478 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.132451    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"kube-root-ca.crt\""
Oct 30 22:41:00.132616 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.132519    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"openshift-service-ca.crt\""
Oct 30 22:41:00.133773 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.133750    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-epp-sa-dockercfg-cphww\""
Oct 30 22:41:00.133873 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.133761    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"scheduler-custom-template-test-kserve-self-signed-certs\""
Oct 30 22:41:00.133873 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.133783    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-35377f35\"/\"default-dockercfg-wtn5p\""
Oct 30 22:41:00.143172 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.143146    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"]
Oct 30 22:41:00.169087 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.169005    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:00.177199 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.177168    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw66g\" (UniqueName: \"kubernetes.io/projected/034891f4-d78a-4cc6-87ae-f2104336d9b6-kube-api-access-rw66g\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.177370 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.177211    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034891f4-d78a-4cc6-87ae-f2104336d9b6-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.177370 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.177236    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.177370 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.177343    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.278150 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.278050    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.278150 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.278136    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw66g\" (UniqueName: \"kubernetes.io/projected/034891f4-d78a-4cc6-87ae-f2104336d9b6-kube-api-access-rw66g\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.278690 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.278163    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034891f4-d78a-4cc6-87ae-f2104336d9b6-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.278690 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.278183    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.278690 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.278525    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-tokenizer-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.278690 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.278527    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-cachi2-cache\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.280815 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.280795    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034891f4-d78a-4cc6-87ae-f2104336d9b6-tls-certs\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.290912 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.290884    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw66g\" (UniqueName: \"kubernetes.io/projected/034891f4-d78a-4cc6-87ae-f2104336d9b6-kube-api-access-rw66g\") pod \"scheduler-custom-template-test-kserve-router-scheduler-bd5744f4\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") " pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.439043 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.439005    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:00.736386 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.736362    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"]
Oct 30 22:41:00.738401 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:41:00.738354    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034891f4_d78a_4cc6_87ae_f2104336d9b6.slice/crio-5b2ab69a8fc5788efa12249f8deb5f771d64cc1fe16b26fc5b1eb66575ec8914 WatchSource:0}: Error finding container 5b2ab69a8fc5788efa12249f8deb5f771d64cc1fe16b26fc5b1eb66575ec8914: Status 404 returned error can't find the container with id 5b2ab69a8fc5788efa12249f8deb5f771d64cc1fe16b26fc5b1eb66575ec8914
Oct 30 22:41:00.963746 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:00.960156    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:01.438748 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:01.438711    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" event={"ID":"034891f4-d78a-4cc6-87ae-f2104336d9b6","Type":"ContainerStarted","Data":"d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21"}
Oct 30 22:41:01.439346 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:01.439317    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" event={"ID":"034891f4-d78a-4cc6-87ae-f2104336d9b6","Type":"ContainerStarted","Data":"5b2ab69a8fc5788efa12249f8deb5f771d64cc1fe16b26fc5b1eb66575ec8914"}
Oct 30 22:41:01.452696 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:01.440321    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:02.476157 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:02.476085    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:04.944698 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:04.944511    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:05.112587 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:05.112549    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:05.215410 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:05.215315    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:05.484806 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:05.484701    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:07.052424 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:07.052370    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:07.273488 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:07.273456    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:11.079098 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:11.079059    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:11.211080 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:11.211037    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:14.891062 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:14.891029    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:15.389633 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:41:15.389599    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:41:19.759932 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:19.759895    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:26.830477 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:26.830442    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:27.432016 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:27.431983    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:33.474996 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:33.474969    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:41:33.505238 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:33.505177    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" podStartSLOduration=33.505160263 podStartE2EDuration="33.505160263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:41:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:41:01.478231018 +0000 UTC m=+1303.747312543" watchObservedRunningTime="2026-10-30 22:41:33.505160263 +0000 UTC m=+1335.774241785"
Oct 30 22:41:35.141682 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:35.141626    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"]
Oct 30 22:41:35.143492 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:35.143454    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:35.244818 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:35.244787    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:41:43.475030 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:43.474984    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" podUID="034891f4-d78a-4cc6-87ae-f2104336d9b6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:41:45.389478 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:41:45.389442    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:41:50.162974 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:50.162904    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" podUID="034891f4-d78a-4cc6-87ae-f2104336d9b6" containerName="main" containerID="cri-o://d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21" gracePeriod=45
Oct 30 22:41:53.474228 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:41:53.474189    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" podUID="034891f4-d78a-4cc6-87ae-f2104336d9b6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:42:03.474407 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:03.474373    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" podUID="034891f4-d78a-4cc6-87ae-f2104336d9b6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:42:03.474995 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:03.474470    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:42:13.474255 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:13.474216    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" podUID="034891f4-d78a-4cc6-87ae-f2104336d9b6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:42:14.944049 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:14.944015    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:15.389151 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:42:15.389120    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:42:19.712535 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:19.712504    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:20.409093 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.409070    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:42:20.521083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.521009    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-cachi2-cache\") pod \"034891f4-d78a-4cc6-87ae-f2104336d9b6\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") "
Oct 30 22:42:20.521083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.521056    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/034891f4-d78a-4cc6-87ae-f2104336d9b6-tls-certs\" (UniqueName: \"kubernetes.io/secret/034891f4-d78a-4cc6-87ae-f2104336d9b6-tls-certs\") pod \"034891f4-d78a-4cc6-87ae-f2104336d9b6\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") "
Oct 30 22:42:20.521083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.521081    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/034891f4-d78a-4cc6-87ae-f2104336d9b6-kube-api-access-rw66g\" (UniqueName: \"kubernetes.io/projected/034891f4-d78a-4cc6-87ae-f2104336d9b6-kube-api-access-rw66g\") pod \"034891f4-d78a-4cc6-87ae-f2104336d9b6\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") "
Oct 30 22:42:20.521308 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.521183    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-tokenizer-cache\") pod \"034891f4-d78a-4cc6-87ae-f2104336d9b6\" (UID: \"034891f4-d78a-4cc6-87ae-f2104336d9b6\") "
Oct 30 22:42:20.521308 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.521295    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-cachi2-cache" pod "034891f4-d78a-4cc6-87ae-f2104336d9b6" (UID: "034891f4-d78a-4cc6-87ae-f2104336d9b6"). InnerVolumeSpecName "cachi2-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:42:20.521463 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.521442    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-tokenizer-cache" pod "034891f4-d78a-4cc6-87ae-f2104336d9b6" (UID: "034891f4-d78a-4cc6-87ae-f2104336d9b6"). InnerVolumeSpecName "tokenizer-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:42:20.521543 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.521529    5503 reconciler_common.go:299] "Volume detached for volume \"tokenizer-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-tokenizer-cache\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:20.521592 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.521547    5503 reconciler_common.go:299] "Volume detached for volume \"cachi2-cache\" (UniqueName: \"kubernetes.io/empty-dir/034891f4-d78a-4cc6-87ae-f2104336d9b6-cachi2-cache\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:20.523340 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.523314    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/034891f4-d78a-4cc6-87ae-f2104336d9b6-tls-certs" pod "034891f4-d78a-4cc6-87ae-f2104336d9b6" (UID: "034891f4-d78a-4cc6-87ae-f2104336d9b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:42:20.523340 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.523330    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/034891f4-d78a-4cc6-87ae-f2104336d9b6-kube-api-access-rw66g" pod "034891f4-d78a-4cc6-87ae-f2104336d9b6" (UID: "034891f4-d78a-4cc6-87ae-f2104336d9b6"). InnerVolumeSpecName "kube-api-access-rw66g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:42:20.622882 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.622860    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/034891f4-d78a-4cc6-87ae-f2104336d9b6-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:20.622882 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.622880    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rw66g\" (UniqueName: \"kubernetes.io/projected/034891f4-d78a-4cc6-87ae-f2104336d9b6-kube-api-access-rw66g\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:42:20.973122 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.973098    5503 generic.go:356] "Generic (PLEG): container finished" podID="034891f4-d78a-4cc6-87ae-f2104336d9b6" containerID="d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21" exitCode=0
Oct 30 22:42:20.973441 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.973144    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"
Oct 30 22:42:20.973441 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.973154    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" event={"ID":"034891f4-d78a-4cc6-87ae-f2104336d9b6","Type":"ContainerDied","Data":"d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21"}
Oct 30 22:42:20.973441 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.973193    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4" event={"ID":"034891f4-d78a-4cc6-87ae-f2104336d9b6","Type":"ContainerDied","Data":"5b2ab69a8fc5788efa12249f8deb5f771d64cc1fe16b26fc5b1eb66575ec8914"}
Oct 30 22:42:20.973441 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.973212    5503 scope.go:122] "RemoveContainer" containerID="d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21"
Oct 30 22:42:20.982259 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.982243    5503 scope.go:122] "RemoveContainer" containerID="d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21"
Oct 30 22:42:20.982514 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:42:20.982494    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21\": container with ID starting with d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21 not found: ID does not exist" containerID="d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21"
Oct 30 22:42:20.982575 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.982521    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21"} err="failed to get container status \"d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21\": rpc error: code = NotFound desc = could not find container \"d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21\": container with ID starting with d5044f4e33f8f1858ff901e26990554051b966435afff7bd3e7f698487623e21 not found: ID does not exist"
Oct 30 22:42:20.997350 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:20.997328    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"]
Oct 30 22:42:21.002002 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:21.001984    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-35377f35/scheduler-custom-template-test-kserve-router-scheduler-bd5744f4"]
Oct 30 22:42:21.855190 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:21.855165    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="034891f4-d78a-4cc6-87ae-f2104336d9b6" path="/var/lib/kubelet/pods/034891f4-d78a-4cc6-87ae-f2104336d9b6/volumes"
Oct 30 22:42:25.539100 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:25.539068    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:25.680369 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:25.680339    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:25.809013 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:25.808947    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:27.860426 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:27.860401    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:28.549193 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:28.549162    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:32.140940 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.140903    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:32.324073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.324033    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:32.374017 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.373987    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"]
Oct 30 22:42:32.374377 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.374365    5503 cpu_manager.go:414] "containerMap: removing container" podUID="034891f4-d78a-4cc6-87ae-f2104336d9b6" containerName="main"
Oct 30 22:42:32.374419 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.374378    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="034891f4-d78a-4cc6-87ae-f2104336d9b6" containerName="main"
Oct 30 22:42:32.374463 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.374454    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="034891f4-d78a-4cc6-87ae-f2104336d9b6" containerName="main"
Oct 30 22:42:32.379034 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.379012    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:42:32.383792 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.383771    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Oct 30 22:42:32.384892 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.384869    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-epp-sa-dockercfg-lkmlf\""
Oct 30 22:42:32.390899 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.390877    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"openshift-service-ca.crt\""
Oct 30 22:42:32.391078 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.391012    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"kube-root-ca.crt\""
Oct 30 22:42:32.393100 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.393073    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"]
Oct 30 22:42:32.447972 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.447948    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:32.483930 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.483912    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"default-dockercfg-vpmlk\""
Oct 30 22:42:32.518524 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.518489    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4gjk\" (UniqueName: \"kubernetes.io/projected/cceff926-e51b-4ab8-8574-0e0ab2242fc2-kube-api-access-p4gjk\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5\" (UID: \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:42:32.518649 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.518542    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cceff926-e51b-4ab8-8574-0e0ab2242fc2-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5\" (UID: \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:42:32.619504 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.619469    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4gjk\" (UniqueName: \"kubernetes.io/projected/cceff926-e51b-4ab8-8574-0e0ab2242fc2-kube-api-access-p4gjk\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5\" (UID: \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:42:32.619504 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.619503    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cceff926-e51b-4ab8-8574-0e0ab2242fc2-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5\" (UID: \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:42:32.622149 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.622125    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cceff926-e51b-4ab8-8574-0e0ab2242fc2-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5\" (UID: \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:42:32.634652 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.634630    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4gjk\" (UniqueName: \"kubernetes.io/projected/cceff926-e51b-4ab8-8574-0e0ab2242fc2-kube-api-access-p4gjk\") pod \"scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5\" (UID: \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:42:32.697697 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.697629    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:42:32.988913 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:32.988875    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"]
Oct 30 22:42:32.991115 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:42:32.991072    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcceff926_e51b_4ab8_8574_0e0ab2242fc2.slice/crio-bf60dc327accfe15190d1c71976bdf3eafe9c0b02244a3ecfaff575702a9ea3e WatchSource:0}: Error finding container bf60dc327accfe15190d1c71976bdf3eafe9c0b02244a3ecfaff575702a9ea3e: Status 404 returned error can't find the container with id bf60dc327accfe15190d1c71976bdf3eafe9c0b02244a3ecfaff575702a9ea3e
Oct 30 22:42:33.077026 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:33.076986    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" event={"ID":"cceff926-e51b-4ab8-8574-0e0ab2242fc2","Type":"ContainerStarted","Data":"bf60dc327accfe15190d1c71976bdf3eafe9c0b02244a3ecfaff575702a9ea3e"}
Oct 30 22:42:34.106688 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:34.105639    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" event={"ID":"cceff926-e51b-4ab8-8574-0e0ab2242fc2","Type":"ContainerStarted","Data":"ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268"}
Oct 30 22:42:34.106688 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:34.106621    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:42:38.623691 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:38.623627    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:38.731334 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:38.731303    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:41.047706 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:41.043104    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:41.737234 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:41.737192    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:45.388697 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:42:45.388636    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:42:59.533304 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:59.533270    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:59.729732 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:59.729702    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:42:59.961305 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:42:59.961279    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:00.063888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:00.063855    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:02.699002 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:02.698968    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:04.593506 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.593456    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" podStartSLOduration=32.593444415 podStartE2EDuration="32.593444415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:42:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:42:34.135189243 +0000 UTC m=+1396.404270766" watchObservedRunningTime="2026-10-30 22:43:04.593444415 +0000 UTC m=+1426.862525938"
Oct 30 22:43:04.597485 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.597456    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"]
Oct 30 22:43:04.601986 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.601955    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.609207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.606268    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"kube-root-ca.crt\""
Oct 30 22:43:04.609207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.606352    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"openshift-service-ca.crt\""
Oct 30 22:43:04.609207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.607377    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"default-dockercfg-72mzm\""
Oct 30 22:43:04.609207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.607484    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-94799d44\"/\"gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs\""
Oct 30 22:43:04.609207 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.608684    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"]
Oct 30 22:43:04.723007 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.722975    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.723172 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.723018    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.723172 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.723042    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009fb0e0-0814-4648-9667-b65990f85a28-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.723172 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.723136    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkxrg\" (UniqueName: \"kubernetes.io/projected/009fb0e0-0814-4648-9667-b65990f85a28-kube-api-access-kkxrg\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.723300 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.723203    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.723300 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.723232    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.824426 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.824394    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009fb0e0-0814-4648-9667-b65990f85a28-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.824630 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.824447    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkxrg\" (UniqueName: \"kubernetes.io/projected/009fb0e0-0814-4648-9667-b65990f85a28-kube-api-access-kkxrg\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.824630 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.824535    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.824630 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.824593    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.824845 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.824633    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.824845 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.824711    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.825029 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.825006    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-model-cache\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.825115 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.825062    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-tmp-dir\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.825198 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.825181    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-home\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.827058 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.827030    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-dshm\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.827203 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.827183    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009fb0e0-0814-4648-9667-b65990f85a28-tls-certs\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.837132 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.837107    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkxrg\" (UniqueName: \"kubernetes.io/projected/009fb0e0-0814-4648-9667-b65990f85a28-kube-api-access-kkxrg\") pod \"gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") " pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:04.921009 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:04.920954    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:05.200000 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:05.199928    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"]
Oct 30 22:43:05.203318 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:43:05.203288    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod009fb0e0_0814_4648_9667_b65990f85a28.slice/crio-b28cdb804bccf1b21cd50fc3c68a57dfe70932307baf170e21a754546fcb4b52 WatchSource:0}: Error finding container b28cdb804bccf1b21cd50fc3c68a57dfe70932307baf170e21a754546fcb4b52: Status 404 returned error can't find the container with id b28cdb804bccf1b21cd50fc3c68a57dfe70932307baf170e21a754546fcb4b52
Oct 30 22:43:05.205419 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:05.205399    5503 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:43:05.334718 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:05.334688    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw" event={"ID":"009fb0e0-0814-4648-9667-b65990f85a28","Type":"ContainerStarted","Data":"b28cdb804bccf1b21cd50fc3c68a57dfe70932307baf170e21a754546fcb4b52"}
Oct 30 22:43:05.513946 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:05.513858    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:06.147171 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:06.147144    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:43:07.348089 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:07.348062    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw" event={"ID":"009fb0e0-0814-4648-9667-b65990f85a28","Type":"ContainerStarted","Data":"0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57"}
Oct 30 22:43:07.369558 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:07.369515    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw" podStartSLOduration=1.664091629 podStartE2EDuration="3.369502089s" totalImagesPullingTime="1.70541046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:43:07.368258321 +0000 UTC m=+1429.637339844" watchObservedRunningTime="2026-10-30 22:43:07.369502089 +0000 UTC m=+1429.638583610"
Oct 30 22:43:07.922003 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:07.921973    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:13.014172 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.014137    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:13.055792 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.055762    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"]
Oct 30 22:43:13.065686 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:43:13.065638    5503 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57" cmd=["/bin/sleep","15"]
Oct 30 22:43:13.065802 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.065716    5503 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:43:13.065802 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:43:13.065729    5503 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw" podUID="009fb0e0-0814-4648-9667-b65990f85a28" containerName="main" containerID="cri-o://0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57"
Oct 30 22:43:13.065802 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.065778    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw" podUID="009fb0e0-0814-4648-9667-b65990f85a28" containerName="main" containerID="cri-o://0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57" gracePeriod=60
Oct 30 22:43:13.073509 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.073483    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"]
Oct 30 22:43:13.148729 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.148696    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:13.324942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.324918    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:13.379295 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.379271    5503 generic.go:356] "Generic (PLEG): container finished" podID="009fb0e0-0814-4648-9667-b65990f85a28" containerID="0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57" exitCode=0
Oct 30 22:43:13.379428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.379369    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"
Oct 30 22:43:13.379428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.379379    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw" event={"ID":"009fb0e0-0814-4648-9667-b65990f85a28","Type":"ContainerDied","Data":"0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57"}
Oct 30 22:43:13.379428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.379426    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw" event={"ID":"009fb0e0-0814-4648-9667-b65990f85a28","Type":"ContainerDied","Data":"b28cdb804bccf1b21cd50fc3c68a57dfe70932307baf170e21a754546fcb4b52"}
Oct 30 22:43:13.379592 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.379453    5503 scope.go:122] "RemoveContainer" containerID="0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57"
Oct 30 22:43:13.391092 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.391043    5503 scope.go:122] "RemoveContainer" containerID="0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57"
Oct 30 22:43:13.391608 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:43:13.391592    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57\": container with ID starting with 0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57 not found: ID does not exist" containerID="0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57"
Oct 30 22:43:13.391695 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.391623    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57"} err="failed to get container status \"0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57\": rpc error: code = NotFound desc = could not find container \"0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57\": container with ID starting with 0c9a3f3ab1255d19d67731ed0eef7459477d2cb1c1b69c595ada51d1a0ffdd57 not found: ID does not exist"
Oct 30 22:43:13.499551 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.499523    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-model-cache\") pod \"009fb0e0-0814-4648-9667-b65990f85a28\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") "
Oct 30 22:43:13.499702 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.499574    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-home\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-home\") pod \"009fb0e0-0814-4648-9667-b65990f85a28\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") "
Oct 30 22:43:13.499702 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.499603    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-dshm\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-dshm\") pod \"009fb0e0-0814-4648-9667-b65990f85a28\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") "
Oct 30 22:43:13.499819 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.499702    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-model-cache" pod "009fb0e0-0814-4648-9667-b65990f85a28" (UID: "009fb0e0-0814-4648-9667-b65990f85a28"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:43:13.499819 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.499775    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/009fb0e0-0814-4648-9667-b65990f85a28-kube-api-access-kkxrg\" (UniqueName: \"kubernetes.io/projected/009fb0e0-0814-4648-9667-b65990f85a28-kube-api-access-kkxrg\") pod \"009fb0e0-0814-4648-9667-b65990f85a28\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") "
Oct 30 22:43:13.499819 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.499806    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-tmp-dir\") pod \"009fb0e0-0814-4648-9667-b65990f85a28\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") "
Oct 30 22:43:13.500288 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.499833    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-home" pod "009fb0e0-0814-4648-9667-b65990f85a28" (UID: "009fb0e0-0814-4648-9667-b65990f85a28"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:43:13.500288 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.499851    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/009fb0e0-0814-4648-9667-b65990f85a28-tls-certs\" (UniqueName: \"kubernetes.io/secret/009fb0e0-0814-4648-9667-b65990f85a28-tls-certs\") pod \"009fb0e0-0814-4648-9667-b65990f85a28\" (UID: \"009fb0e0-0814-4648-9667-b65990f85a28\") "
Oct 30 22:43:13.500288 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.500006    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-tmp-dir" pod "009fb0e0-0814-4648-9667-b65990f85a28" (UID: "009fb0e0-0814-4648-9667-b65990f85a28"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:43:13.500288 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.500231    5503 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-model-cache\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.500466 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.500265    5503 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-home\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.500466 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.500394    5503 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-tmp-dir\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.502411 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.502381    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/009fb0e0-0814-4648-9667-b65990f85a28-tls-certs" pod "009fb0e0-0814-4648-9667-b65990f85a28" (UID: "009fb0e0-0814-4648-9667-b65990f85a28"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:43:13.503827 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.503757    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-dshm" pod "009fb0e0-0814-4648-9667-b65990f85a28" (UID: "009fb0e0-0814-4648-9667-b65990f85a28"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:43:13.524292 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.515271    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/009fb0e0-0814-4648-9667-b65990f85a28-kube-api-access-kkxrg" pod "009fb0e0-0814-4648-9667-b65990f85a28" (UID: "009fb0e0-0814-4648-9667-b65990f85a28"). InnerVolumeSpecName "kube-api-access-kkxrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:43:13.600949 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.600922    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkxrg\" (UniqueName: \"kubernetes.io/projected/009fb0e0-0814-4648-9667-b65990f85a28-kube-api-access-kkxrg\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.600949 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.600947    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009fb0e0-0814-4648-9667-b65990f85a28-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.601128 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.600959    5503 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009fb0e0-0814-4648-9667-b65990f85a28-dshm\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.702501 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.702471    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"]
Oct 30 22:43:13.707640 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.707616    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-94799d44/gw-section-name-router-with-gat-f1d92d0f-kserve-cf8f45b46-58xcw"]
Oct 30 22:43:13.855497 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:13.855442    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="009fb0e0-0814-4648-9667-b65990f85a28" path="/var/lib/kubelet/pods/009fb0e0-0814-4648-9667-b65990f85a28/volumes"
Oct 30 22:43:14.995806 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:14.995780    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:15.389380 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:43:15.389348    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:43:16.145004 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:16.144974    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:43:16.541187 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:16.541128    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:16.642964 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:16.642942    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:19.865932 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:19.865896    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:43:26.145139 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:26.145107    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:43:28.093401 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:28.093300    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerName="main" containerID="cri-o://ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268" gracePeriod=45
Oct 30 22:43:36.145735 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:36.145701    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:43:36.146162 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:36.145810    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:43:45.388191 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:43:45.388154    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:43:46.145492 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:46.145460    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:43:56.145333 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:56.145298    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:43:58.347092 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.347070    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:43:58.438387 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.438325    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cceff926-e51b-4ab8-8574-0e0ab2242fc2-tls-certs\" (UniqueName: \"kubernetes.io/secret/cceff926-e51b-4ab8-8574-0e0ab2242fc2-tls-certs\") pod \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\" (UID: \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\") "
Oct 30 22:43:58.438516 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.438414    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cceff926-e51b-4ab8-8574-0e0ab2242fc2-kube-api-access-p4gjk\" (UniqueName: \"kubernetes.io/projected/cceff926-e51b-4ab8-8574-0e0ab2242fc2-kube-api-access-p4gjk\") pod \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\" (UID: \"cceff926-e51b-4ab8-8574-0e0ab2242fc2\") "
Oct 30 22:43:58.440643 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.440615    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cceff926-e51b-4ab8-8574-0e0ab2242fc2-tls-certs" pod "cceff926-e51b-4ab8-8574-0e0ab2242fc2" (UID: "cceff926-e51b-4ab8-8574-0e0ab2242fc2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:43:58.440759 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.440644    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cceff926-e51b-4ab8-8574-0e0ab2242fc2-kube-api-access-p4gjk" pod "cceff926-e51b-4ab8-8574-0e0ab2242fc2" (UID: "cceff926-e51b-4ab8-8574-0e0ab2242fc2"). InnerVolumeSpecName "kube-api-access-p4gjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:43:58.539339 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.539307    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4gjk\" (UniqueName: \"kubernetes.io/projected/cceff926-e51b-4ab8-8574-0e0ab2242fc2-kube-api-access-p4gjk\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:58.539339 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.539332    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cceff926-e51b-4ab8-8574-0e0ab2242fc2-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:58.595167 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.595138    5503 generic.go:356] "Generic (PLEG): container finished" podID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerID="ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268" exitCode=0
Oct 30 22:43:58.595277 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.595180    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"
Oct 30 22:43:58.595277 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.595180    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" event={"ID":"cceff926-e51b-4ab8-8574-0e0ab2242fc2","Type":"ContainerDied","Data":"ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268"}
Oct 30 22:43:58.595348 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.595307    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5" event={"ID":"cceff926-e51b-4ab8-8574-0e0ab2242fc2","Type":"ContainerDied","Data":"bf60dc327accfe15190d1c71976bdf3eafe9c0b02244a3ecfaff575702a9ea3e"}
Oct 30 22:43:58.595383 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.595348    5503 scope.go:122] "RemoveContainer" containerID="ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268"
Oct 30 22:43:58.607322 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.607232    5503 scope.go:122] "RemoveContainer" containerID="ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268"
Oct 30 22:43:58.608645 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:43:58.608598    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268\": container with ID starting with ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268 not found: ID does not exist" containerID="ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268"
Oct 30 22:43:58.608842 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.608810    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268"} err="failed to get container status \"ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268\": rpc error: code = NotFound desc = could not find container \"ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268\": container with ID starting with ccf867ecb1edf1247e0854c6301b564cb03a37495290ae5d35374e7f759db268 not found: ID does not exist"
Oct 30 22:43:58.624633 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.624607    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"]
Oct 30 22:43:58.630742 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:58.630720    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-router-scheduler-69vkpw5"]
Oct 30 22:43:59.855369 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:43:59.855331    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" path="/var/lib/kubelet/pods/cceff926-e51b-4ab8-8574-0e0ab2242fc2/volumes"
Oct 30 22:44:03.218211 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:03.218177    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:03.321639 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:03.321607    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:03.484273 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:03.484209    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:05.971596 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:05.971553    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:06.138553 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.138515    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"]
Oct 30 22:44:06.139290 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.139257    5503 cpu_manager.go:414] "containerMap: removing container" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerName="main"
Oct 30 22:44:06.139290 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.139284    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerName="main"
Oct 30 22:44:06.139442 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.139312    5503 cpu_manager.go:414] "containerMap: removing container" podUID="009fb0e0-0814-4648-9667-b65990f85a28" containerName="main"
Oct 30 22:44:06.139442 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.139321    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="009fb0e0-0814-4648-9667-b65990f85a28" containerName="main"
Oct 30 22:44:06.139442 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.139412    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="cceff926-e51b-4ab8-8574-0e0ab2242fc2" containerName="main"
Oct 30 22:44:06.139442 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.139426    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="009fb0e0-0814-4648-9667-b65990f85a28" containerName="main"
Oct 30 22:44:06.142643 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.142626    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:06.147834 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.147810    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Oct 30 22:44:06.148932 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.148902    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"]
Oct 30 22:44:06.149987 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.149968    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"openshift-service-ca.crt\""
Oct 30 22:44:06.152175 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.152148    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"kube-root-ca.crt\""
Oct 30 22:44:06.199781 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.199757    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvf72\" (UniqueName: \"kubernetes.io/projected/0d883616-148f-431b-ae97-156462e54210-kube-api-access-qvf72\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp\" (UID: \"0d883616-148f-431b-ae97-156462e54210\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:06.199971 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.199952    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d883616-148f-431b-ae97-156462e54210-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp\" (UID: \"0d883616-148f-431b-ae97-156462e54210\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:06.247888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.247831    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-epp-sa-dockercfg-5dnvn\""
Oct 30 22:44:06.247888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.247859    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"default-dockercfg-b4s4l\""
Oct 30 22:44:06.300896 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.300865    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d883616-148f-431b-ae97-156462e54210-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp\" (UID: \"0d883616-148f-431b-ae97-156462e54210\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:06.300993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.300910    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvf72\" (UniqueName: \"kubernetes.io/projected/0d883616-148f-431b-ae97-156462e54210-kube-api-access-qvf72\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp\" (UID: \"0d883616-148f-431b-ae97-156462e54210\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:06.303465 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.303430    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d883616-148f-431b-ae97-156462e54210-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp\" (UID: \"0d883616-148f-431b-ae97-156462e54210\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:06.314056 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.314031    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvf72\" (UniqueName: \"kubernetes.io/projected/0d883616-148f-431b-ae97-156462e54210-kube-api-access-qvf72\") pod \"scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp\" (UID: \"0d883616-148f-431b-ae97-156462e54210\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:06.458135 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.458101    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:06.714688 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.713908    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:06.767889 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:06.763164    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"]
Oct 30 22:44:07.677338 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:07.677301    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" event={"ID":"0d883616-148f-431b-ae97-156462e54210","Type":"ContainerStarted","Data":"6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c"}
Oct 30 22:44:07.688096 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:07.688070    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:07.688245 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:07.688227    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" event={"ID":"0d883616-148f-431b-ae97-156462e54210","Type":"ContainerStarted","Data":"c1254f1dcd221094c352e4f31a6791ca9cb9510755ca5d227006cc6c565827d3"}
Oct 30 22:44:07.708692 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:07.707652    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" podStartSLOduration=1.707636145 podStartE2EDuration="1.707636145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:44:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:44:07.705871905 +0000 UTC m=+1489.974953429" watchObservedRunningTime="2026-10-30 22:44:07.707636145 +0000 UTC m=+1489.976717672"
Oct 30 22:44:10.202871 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:10.202840    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:15.047066 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:15.047034    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:15.388304 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:44:15.388273    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:44:15.915905 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:15.915876    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:16.012888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:16.012861    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:18.803786 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:18.803753    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:19.918238 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:19.918205    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:20.104052 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:20.104018    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:20.204629 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:20.204565    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:38.699161 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:38.699136    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:44:39.807788 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:39.807754    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:41.261509 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:41.261478    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"]
Oct 30 22:44:41.266705 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:41.265619    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:41.376339 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:41.376307    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:44:45.388442 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:44:45.388408    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:44:48.699307 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:48.699270    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" podUID="0d883616-148f-431b-ae97-156462e54210" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:44:56.288748 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:56.288692    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" podUID="0d883616-148f-431b-ae97-156462e54210" containerName="main" containerID="cri-o://6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c" gracePeriod=45
Oct 30 22:44:58.699180 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:44:58.699136    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" podUID="0d883616-148f-431b-ae97-156462e54210" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:45:08.699166 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:08.699132    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" podUID="0d883616-148f-431b-ae97-156462e54210" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:45:08.699575 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:08.699245    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:45:15.120463 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:15.120432    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:15.388800 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:45:15.388728    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:45:18.698994 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:18.698965    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" podUID="0d883616-148f-431b-ae97-156462e54210" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:45:18.772230 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:18.772200    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:18.946123 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:18.946092    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:19.063808 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:19.063743    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:19.979592 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:19.979566    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:23.809279 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:23.809245    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:24.284397 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.284366    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:24.493844 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.493806    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"]
Oct 30 22:45:24.497417 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.497392    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:24.504385 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.503920    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"kube-root-ca.crt\""
Oct 30 22:45:24.505176 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.505154    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"default-dockercfg-5q6lw\""
Oct 30 22:45:24.505176 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.505177    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-epp-sa-dockercfg-8vsnp\""
Oct 30 22:45:24.505799 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.505777    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"auth-enabled-test-kserve-self-signed-certs\""
Oct 30 22:45:24.507592 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.507567    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-enabled-requires-token-d4ed2c08\"/\"openshift-service-ca.crt\""
Oct 30 22:45:24.516651 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.516629    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"]
Oct 30 22:45:24.570468 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.570405    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:24.572579 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.572560    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56767a23-a7b9-4418-8516-692469a7bcce-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w\" (UID: \"56767a23-a7b9-4418-8516-692469a7bcce\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:24.572687 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.572590    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqbcq\" (UniqueName: \"kubernetes.io/projected/56767a23-a7b9-4418-8516-692469a7bcce-kube-api-access-jqbcq\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w\" (UID: \"56767a23-a7b9-4418-8516-692469a7bcce\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:24.673369 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.673345    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56767a23-a7b9-4418-8516-692469a7bcce-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w\" (UID: \"56767a23-a7b9-4418-8516-692469a7bcce\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:24.673470 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.673376    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqbcq\" (UniqueName: \"kubernetes.io/projected/56767a23-a7b9-4418-8516-692469a7bcce-kube-api-access-jqbcq\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w\" (UID: \"56767a23-a7b9-4418-8516-692469a7bcce\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:24.675991 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.675970    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56767a23-a7b9-4418-8516-692469a7bcce-tls-certs\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w\" (UID: \"56767a23-a7b9-4418-8516-692469a7bcce\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:24.688073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.688055    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqbcq\" (UniqueName: \"kubernetes.io/projected/56767a23-a7b9-4418-8516-692469a7bcce-kube-api-access-jqbcq\") pod \"auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w\" (UID: \"56767a23-a7b9-4418-8516-692469a7bcce\") " pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:24.810802 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:24.810745    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:25.103213 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:25.102869    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"]
Oct 30 22:45:26.116819 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.116783    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" event={"ID":"56767a23-a7b9-4418-8516-692469a7bcce","Type":"ContainerStarted","Data":"172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7"}
Oct 30 22:45:26.117237 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.116840    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" event={"ID":"56767a23-a7b9-4418-8516-692469a7bcce","Type":"ContainerStarted","Data":"327587da7c495b38c4cbb8d3ce39c8ac86b8f0f23a3930091cea59d4c608d53b"}
Oct 30 22:45:26.117237 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.116897    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:26.138015 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.137964    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" podStartSLOduration=2.137951955 podStartE2EDuration="2.137951955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:45:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:45:26.136001453 +0000 UTC m=+1568.405082980" watchObservedRunningTime="2026-10-30 22:45:26.137951955 +0000 UTC m=+1568.407033477"
Oct 30 22:45:26.537286 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.537265    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:45:26.596364 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.596339    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d883616-148f-431b-ae97-156462e54210-tls-certs\" (UniqueName: \"kubernetes.io/secret/0d883616-148f-431b-ae97-156462e54210-tls-certs\") pod \"0d883616-148f-431b-ae97-156462e54210\" (UID: \"0d883616-148f-431b-ae97-156462e54210\") "
Oct 30 22:45:26.596488 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.596393    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d883616-148f-431b-ae97-156462e54210-kube-api-access-qvf72\" (UniqueName: \"kubernetes.io/projected/0d883616-148f-431b-ae97-156462e54210-kube-api-access-qvf72\") pod \"0d883616-148f-431b-ae97-156462e54210\" (UID: \"0d883616-148f-431b-ae97-156462e54210\") "
Oct 30 22:45:26.598651 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.598618    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d883616-148f-431b-ae97-156462e54210-kube-api-access-qvf72" pod "0d883616-148f-431b-ae97-156462e54210" (UID: "0d883616-148f-431b-ae97-156462e54210"). InnerVolumeSpecName "kube-api-access-qvf72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:45:26.598651 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.598630    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d883616-148f-431b-ae97-156462e54210-tls-certs" pod "0d883616-148f-431b-ae97-156462e54210" (UID: "0d883616-148f-431b-ae97-156462e54210"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:45:26.697706 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.697649    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvf72\" (UniqueName: \"kubernetes.io/projected/0d883616-148f-431b-ae97-156462e54210-kube-api-access-qvf72\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:45:26.697706 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:26.697681    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d883616-148f-431b-ae97-156462e54210-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:45:27.124079 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.124050    5503 generic.go:356] "Generic (PLEG): container finished" podID="0d883616-148f-431b-ae97-156462e54210" containerID="6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c" exitCode=0
Oct 30 22:45:27.124587 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.124114    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"
Oct 30 22:45:27.124587 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.124134    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" event={"ID":"0d883616-148f-431b-ae97-156462e54210","Type":"ContainerDied","Data":"6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c"}
Oct 30 22:45:27.124587 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.124189    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp" event={"ID":"0d883616-148f-431b-ae97-156462e54210","Type":"ContainerDied","Data":"c1254f1dcd221094c352e4f31a6791ca9cb9510755ca5d227006cc6c565827d3"}
Oct 30 22:45:27.124587 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.124218    5503 scope.go:122] "RemoveContainer" containerID="6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c"
Oct 30 22:45:27.135801 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.135641    5503 scope.go:122] "RemoveContainer" containerID="6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c"
Oct 30 22:45:27.136526 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:45:27.136493    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c\": container with ID starting with 6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c not found: ID does not exist" containerID="6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c"
Oct 30 22:45:27.136636 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.136534    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c"} err="failed to get container status \"6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c\": rpc error: code = NotFound desc = could not find container \"6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c\": container with ID starting with 6bc3feb8d5992544c8d5efe035c8b545a4a1ed809a9a2bd943c2b9a1ceed162c not found: ID does not exist"
Oct 30 22:45:27.148319 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.148296    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"]
Oct 30 22:45:27.156547 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.156525    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-router-schedqm9jp"]
Oct 30 22:45:27.854832 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:27.854809    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d883616-148f-431b-ae97-156462e54210" path="/var/lib/kubelet/pods/0d883616-148f-431b-ae97-156462e54210/volumes"
Oct 30 22:45:29.820116 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:29.820077    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:31.481853 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:31.481818    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:31.593292 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:31.593268    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:31.694188 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:31.694162    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:33.792520 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:33.792489    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:34.142326 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.142292    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"]
Oct 30 22:45:34.142794 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.142776    5503 cpu_manager.go:414] "containerMap: removing container" podUID="0d883616-148f-431b-ae97-156462e54210" containerName="main"
Oct 30 22:45:34.142794 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.142795    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d883616-148f-431b-ae97-156462e54210" containerName="main"
Oct 30 22:45:34.142958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.142932    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d883616-148f-431b-ae97-156462e54210" containerName="main"
Oct 30 22:45:34.147963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.147940    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:45:34.156972 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.156403    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Oct 30 22:45:34.156972 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.156518    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Oct 30 22:45:34.159402 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.159375    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"]
Oct 30 22:45:34.168543 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.168519    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:34.212542 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.212502    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-jwx64\""
Oct 30 22:45:34.212820 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.212800    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Oct 30 22:45:34.212908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.212825    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-epp-sa-dockercfg-b5495\""
Oct 30 22:45:34.273600 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.270857    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:34.275045 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.275011    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qntd9\" (UniqueName: \"kubernetes.io/projected/04251af1-696f-4766-a6bf-1f0f5aca84e6-kube-api-access-qntd9\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j\" (UID: \"04251af1-696f-4766-a6bf-1f0f5aca84e6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:45:34.275213 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.275175    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04251af1-696f-4766-a6bf-1f0f5aca84e6-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j\" (UID: \"04251af1-696f-4766-a6bf-1f0f5aca84e6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:45:34.376432 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.376403    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04251af1-696f-4766-a6bf-1f0f5aca84e6-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j\" (UID: \"04251af1-696f-4766-a6bf-1f0f5aca84e6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:45:34.376608 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.376455    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qntd9\" (UniqueName: \"kubernetes.io/projected/04251af1-696f-4766-a6bf-1f0f5aca84e6-kube-api-access-qntd9\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j\" (UID: \"04251af1-696f-4766-a6bf-1f0f5aca84e6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:45:34.379146 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.379123    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04251af1-696f-4766-a6bf-1f0f5aca84e6-tls-certs\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j\" (UID: \"04251af1-696f-4766-a6bf-1f0f5aca84e6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:45:34.389322 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.389296    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qntd9\" (UniqueName: \"kubernetes.io/projected/04251af1-696f-4766-a6bf-1f0f5aca84e6-kube-api-access-qntd9\") pod \"tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j\" (UID: \"04251af1-696f-4766-a6bf-1f0f5aca84e6\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:45:34.464571 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.463935    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:45:34.762556 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:34.762527    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"]
Oct 30 22:45:34.764497 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:45:34.764472    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04251af1_696f_4766_a6bf_1f0f5aca84e6.slice/crio-0bcec41d52bb2a21f8eb0b6f27043adb8627e220337793345bfc72d9285493e4 WatchSource:0}: Error finding container 0bcec41d52bb2a21f8eb0b6f27043adb8627e220337793345bfc72d9285493e4: Status 404 returned error can't find the container with id 0bcec41d52bb2a21f8eb0b6f27043adb8627e220337793345bfc72d9285493e4
Oct 30 22:45:35.171409 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:35.171377    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" event={"ID":"04251af1-696f-4766-a6bf-1f0f5aca84e6","Type":"ContainerStarted","Data":"ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e"}
Oct 30 22:45:35.171826 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:35.171429    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" event={"ID":"04251af1-696f-4766-a6bf-1f0f5aca84e6","Type":"ContainerStarted","Data":"0bcec41d52bb2a21f8eb0b6f27043adb8627e220337793345bfc72d9285493e4"}
Oct 30 22:45:35.171826 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:35.171504    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:45:37.494646 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:37.494616    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:38.409387 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:38.409303    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:39.862235 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:39.862197    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:44.232990 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:44.232955    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:44.341946 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:44.341882    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:44.972309 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:44.972279    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:45.389060 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:45:45.389025    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:45:46.347881 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:46.347853    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:56.313977 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:56.313940    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:45:57.126007 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:57.125978    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:45:57.147535 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:45:57.147489    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" podStartSLOduration=23.14747528 podStartE2EDuration="23.14747528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:45:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:45:35.194342654 +0000 UTC m=+1577.463424178" watchObservedRunningTime="2026-10-30 22:45:57.14747528 +0000 UTC m=+1599.416556802"
Oct 30 22:46:01.449529 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:01.449493    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:01.566432 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:01.566406    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:06.179422 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:06.179394    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:46:11.738805 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:11.738766    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:11.751197 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:11.751170    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"]
Oct 30 22:46:11.841264 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:11.841237    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:15.151500 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:15.151468    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:15.388589 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:46:15.388557    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:46:16.178637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:16.178596    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:46:26.179178 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:26.179146    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:46:26.771099 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:26.771066    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerName="main" containerID="cri-o://ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e" gracePeriod=45
Oct 30 22:46:36.178764 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:36.178727    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:46:36.179223 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:36.178826    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:46:37.620681 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:37.620627    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:39.917414 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:39.917384    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:46:45.389002 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:46:45.388968    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:46:46.178543 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:46.178514    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:46:56.179237 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:56.179203    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:46:57.009654 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.009625    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:46:57.140297 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.140271    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/04251af1-696f-4766-a6bf-1f0f5aca84e6-tls-certs\" (UniqueName: \"kubernetes.io/secret/04251af1-696f-4766-a6bf-1f0f5aca84e6-tls-certs\") pod \"04251af1-696f-4766-a6bf-1f0f5aca84e6\" (UID: \"04251af1-696f-4766-a6bf-1f0f5aca84e6\") "
Oct 30 22:46:57.140420 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.140319    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04251af1-696f-4766-a6bf-1f0f5aca84e6-kube-api-access-qntd9\" (UniqueName: \"kubernetes.io/projected/04251af1-696f-4766-a6bf-1f0f5aca84e6-kube-api-access-qntd9\") pod \"04251af1-696f-4766-a6bf-1f0f5aca84e6\" (UID: \"04251af1-696f-4766-a6bf-1f0f5aca84e6\") "
Oct 30 22:46:57.142497 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.142469    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04251af1-696f-4766-a6bf-1f0f5aca84e6-tls-certs" pod "04251af1-696f-4766-a6bf-1f0f5aca84e6" (UID: "04251af1-696f-4766-a6bf-1f0f5aca84e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:46:57.142594 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.142497    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04251af1-696f-4766-a6bf-1f0f5aca84e6-kube-api-access-qntd9" pod "04251af1-696f-4766-a6bf-1f0f5aca84e6" (UID: "04251af1-696f-4766-a6bf-1f0f5aca84e6"). InnerVolumeSpecName "kube-api-access-qntd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:46:57.241601 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.241576    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04251af1-696f-4766-a6bf-1f0f5aca84e6-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:57.241601 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.241600    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qntd9\" (UniqueName: \"kubernetes.io/projected/04251af1-696f-4766-a6bf-1f0f5aca84e6-kube-api-access-qntd9\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:57.573130 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.573070    5503 generic.go:356] "Generic (PLEG): container finished" podID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerID="ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e" exitCode=0
Oct 30 22:46:57.573130 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.573115    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"
Oct 30 22:46:57.573271 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.573129    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" event={"ID":"04251af1-696f-4766-a6bf-1f0f5aca84e6","Type":"ContainerDied","Data":"ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e"}
Oct 30 22:46:57.573271 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.573170    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j" event={"ID":"04251af1-696f-4766-a6bf-1f0f5aca84e6","Type":"ContainerDied","Data":"0bcec41d52bb2a21f8eb0b6f27043adb8627e220337793345bfc72d9285493e4"}
Oct 30 22:46:57.573271 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.573192    5503 scope.go:122] "RemoveContainer" containerID="ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e"
Oct 30 22:46:57.582808 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.582791    5503 scope.go:122] "RemoveContainer" containerID="ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e"
Oct 30 22:46:57.583049 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:46:57.583033    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e\": container with ID starting with ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e not found: ID does not exist" containerID="ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e"
Oct 30 22:46:57.583105 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.583056    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e"} err="failed to get container status \"ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e\": rpc error: code = NotFound desc = could not find container \"ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e\": container with ID starting with ece534f7a1472a081f175995acb772ed7e6aa2be075c325d70e30894bfe5508e not found: ID does not exist"
Oct 30 22:46:57.600491 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.600469    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"]
Oct 30 22:46:57.602111 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.602084    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-router-scheduler-84d6588cw2k6j"]
Oct 30 22:46:57.854971 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:46:57.854948    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" path="/var/lib/kubelet/pods/04251af1-696f-4766-a6bf-1f0f5aca84e6/volumes"
Oct 30 22:47:02.096563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:02.096535    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:02.201149 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:02.201118    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:08.327110 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.327078    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:08.540736 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.540698    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"]
Oct 30 22:47:08.541363 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.541328    5503 cpu_manager.go:414] "containerMap: removing container" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerName="main"
Oct 30 22:47:08.541544 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.541520    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerName="main"
Oct 30 22:47:08.541703 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.541689    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="04251af1-696f-4766-a6bf-1f0f5aca84e6" containerName="main"
Oct 30 22:47:08.545200 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.545181    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:08.555472 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.555450    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-9s22j\""
Oct 30 22:47:08.555606 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.555534    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Oct 30 22:47:08.555687 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.555648    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Oct 30 22:47:08.556860 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.556836    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-epp-sa-dockercfg-hb7fs\""
Oct 30 22:47:08.556987 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.556963    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Oct 30 22:47:08.557908 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.557891    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"]
Oct 30 22:47:08.568812 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.568768    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:08.673737 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.673711    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:08.734655 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.734626    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68mr7\" (UniqueName: \"kubernetes.io/projected/4586f536-8969-4d57-98c1-aa015afcf345-kube-api-access-68mr7\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9\" (UID: \"4586f536-8969-4d57-98c1-aa015afcf345\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:08.734797 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.734690    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4586f536-8969-4d57-98c1-aa015afcf345-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9\" (UID: \"4586f536-8969-4d57-98c1-aa015afcf345\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:08.835662 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.835636    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68mr7\" (UniqueName: \"kubernetes.io/projected/4586f536-8969-4d57-98c1-aa015afcf345-kube-api-access-68mr7\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9\" (UID: \"4586f536-8969-4d57-98c1-aa015afcf345\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:08.835817 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.835704    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4586f536-8969-4d57-98c1-aa015afcf345-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9\" (UID: \"4586f536-8969-4d57-98c1-aa015afcf345\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:08.838263 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.838244    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4586f536-8969-4d57-98c1-aa015afcf345-tls-certs\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9\" (UID: \"4586f536-8969-4d57-98c1-aa015afcf345\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:08.848437 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.848408    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68mr7\" (UniqueName: \"kubernetes.io/projected/4586f536-8969-4d57-98c1-aa015afcf345-kube-api-access-68mr7\") pod \"tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9\" (UID: \"4586f536-8969-4d57-98c1-aa015afcf345\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:08.859822 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:08.859795    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:09.119065 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:09.119035    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:09.155950 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:09.155928    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"]
Oct 30 22:47:09.157769 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:47:09.157742    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4586f536_8969_4d57_98c1_aa015afcf345.slice/crio-3e0cc81dbf911f33b2278447d93209f8f6913afc535b07898323d2a96e8849bf WatchSource:0}: Error finding container 3e0cc81dbf911f33b2278447d93209f8f6913afc535b07898323d2a96e8849bf: Status 404 returned error can't find the container with id 3e0cc81dbf911f33b2278447d93209f8f6913afc535b07898323d2a96e8849bf
Oct 30 22:47:09.635386 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:09.635354    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" event={"ID":"4586f536-8969-4d57-98c1-aa015afcf345","Type":"ContainerStarted","Data":"9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0"}
Oct 30 22:47:09.635860 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:09.635409    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" event={"ID":"4586f536-8969-4d57-98c1-aa015afcf345","Type":"ContainerStarted","Data":"3e0cc81dbf911f33b2278447d93209f8f6913afc535b07898323d2a96e8849bf"}
Oct 30 22:47:09.635860 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:09.635494    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:09.656276 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:09.656234    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" podStartSLOduration=1.656221594 podStartE2EDuration="1.656221594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:47:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:47:09.655278755 +0000 UTC m=+1671.924360290" watchObservedRunningTime="2026-10-30 22:47:09.656221594 +0000 UTC m=+1671.925303117"
Oct 30 22:47:11.261858 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:11.261826    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:14.808795 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:14.808758    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:14.910951 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:14.910922    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:15.265644 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:15.265618    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:15.388318 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:47:15.388288    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:47:25.039644 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:25.039607    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:30.164637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:30.164609    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:30.181950 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:30.181919    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"]
Oct 30 22:47:30.272399 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:30.271797    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:37.126487 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:37.126450    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" podUID="56767a23-a7b9-4418-8516-692469a7bcce" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:47:39.974864 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:39.974829    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:40.642107 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:40.642079    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:47:45.209053 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:45.208994    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" podUID="56767a23-a7b9-4418-8516-692469a7bcce" containerName="main" containerID="cri-o://172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7" gracePeriod=45
Oct 30 22:47:45.391020 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:47:45.390977    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:47:46.483035 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:46.483005    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:46.499394 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:46.499354    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"]
Oct 30 22:47:46.596195 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:46.596165    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:47:47.126409 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:47.126371    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" podUID="56767a23-a7b9-4418-8516-692469a7bcce" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:47:50.642075 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:50.642034    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" podUID="4586f536-8969-4d57-98c1-aa015afcf345" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:47:57.125896 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:57.125815    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" podUID="56767a23-a7b9-4418-8516-692469a7bcce" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:47:57.126263 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:47:57.125939    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:48:00.641926 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:00.641888    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" podUID="4586f536-8969-4d57-98c1-aa015afcf345" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:48:01.521298 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:01.521244    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" podUID="4586f536-8969-4d57-98c1-aa015afcf345" containerName="main" containerID="cri-o://9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0" gracePeriod=45
Oct 30 22:48:07.126067 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:07.126031    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" podUID="56767a23-a7b9-4418-8516-692469a7bcce" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:48:10.642587 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:10.642547    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" podUID="4586f536-8969-4d57-98c1-aa015afcf345" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:48:10.643109 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:10.642694    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:48:15.391695 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:48:15.391627    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:48:15.469525 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.469472    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:48:15.505534 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.505510    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/56767a23-a7b9-4418-8516-692469a7bcce-tls-certs\" (UniqueName: \"kubernetes.io/secret/56767a23-a7b9-4418-8516-692469a7bcce-tls-certs\") pod \"56767a23-a7b9-4418-8516-692469a7bcce\" (UID: \"56767a23-a7b9-4418-8516-692469a7bcce\") "
Oct 30 22:48:15.505897 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.505607    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56767a23-a7b9-4418-8516-692469a7bcce-kube-api-access-jqbcq\" (UniqueName: \"kubernetes.io/projected/56767a23-a7b9-4418-8516-692469a7bcce-kube-api-access-jqbcq\") pod \"56767a23-a7b9-4418-8516-692469a7bcce\" (UID: \"56767a23-a7b9-4418-8516-692469a7bcce\") "
Oct 30 22:48:15.507790 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.507766    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56767a23-a7b9-4418-8516-692469a7bcce-kube-api-access-jqbcq" pod "56767a23-a7b9-4418-8516-692469a7bcce" (UID: "56767a23-a7b9-4418-8516-692469a7bcce"). InnerVolumeSpecName "kube-api-access-jqbcq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:15.507790 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.507776    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56767a23-a7b9-4418-8516-692469a7bcce-tls-certs" pod "56767a23-a7b9-4418-8516-692469a7bcce" (UID: "56767a23-a7b9-4418-8516-692469a7bcce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:48:15.607032 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.607007    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqbcq\" (UniqueName: \"kubernetes.io/projected/56767a23-a7b9-4418-8516-692469a7bcce-kube-api-access-jqbcq\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.607032 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.607030    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/56767a23-a7b9-4418-8516-692469a7bcce-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:15.991107 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.991078    5503 generic.go:356] "Generic (PLEG): container finished" podID="56767a23-a7b9-4418-8516-692469a7bcce" containerID="172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7" exitCode=0
Oct 30 22:48:15.991257 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.991115    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"
Oct 30 22:48:15.991257 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.991161    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" event={"ID":"56767a23-a7b9-4418-8516-692469a7bcce","Type":"ContainerDied","Data":"172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7"}
Oct 30 22:48:15.991257 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.991214    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w" event={"ID":"56767a23-a7b9-4418-8516-692469a7bcce","Type":"ContainerDied","Data":"327587da7c495b38c4cbb8d3ce39c8ac86b8f0f23a3930091cea59d4c608d53b"}
Oct 30 22:48:15.991257 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:15.991244    5503 scope.go:122] "RemoveContainer" containerID="172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7"
Oct 30 22:48:16.000320 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:16.000306    5503 scope.go:122] "RemoveContainer" containerID="172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7"
Oct 30 22:48:16.000573 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:48:16.000551    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7\": container with ID starting with 172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7 not found: ID does not exist" containerID="172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7"
Oct 30 22:48:16.000620 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:16.000583    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7"} err="failed to get container status \"172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7\": rpc error: code = NotFound desc = could not find container \"172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7\": container with ID starting with 172ebf14e98e8ad16ed763835fff1eaa2746eba6615ac0da8d6e5446cdc568f7 not found: ID does not exist"
Oct 30 22:48:16.010194 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:16.010175    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"]
Oct 30 22:48:16.016189 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:16.016169    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-enabled-requires-token-d4ed2c08/auth-enabled-test-kserve-router-scheduler-674b8746cc-8c46w"]
Oct 30 22:48:17.855386 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:17.855358    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56767a23-a7b9-4418-8516-692469a7bcce" path="/var/lib/kubelet/pods/56767a23-a7b9-4418-8516-692469a7bcce/volumes"
Oct 30 22:48:20.015041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:20.015005    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:20.126474 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:20.126443    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:20.642961 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:20.642915    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" podUID="4586f536-8969-4d57-98c1-aa015afcf345" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:48:25.100608 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:25.100574    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:26.314677 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:26.314630    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:30.642797 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:30.642751    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" podUID="4586f536-8969-4d57-98c1-aa015afcf345" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:48:31.777360 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:31.777330    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:48:31.840224 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:31.840144    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4586f536-8969-4d57-98c1-aa015afcf345-kube-api-access-68mr7\" (UniqueName: \"kubernetes.io/projected/4586f536-8969-4d57-98c1-aa015afcf345-kube-api-access-68mr7\") pod \"4586f536-8969-4d57-98c1-aa015afcf345\" (UID: \"4586f536-8969-4d57-98c1-aa015afcf345\") "
Oct 30 22:48:31.840224 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:31.840212    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4586f536-8969-4d57-98c1-aa015afcf345-tls-certs\" (UniqueName: \"kubernetes.io/secret/4586f536-8969-4d57-98c1-aa015afcf345-tls-certs\") pod \"4586f536-8969-4d57-98c1-aa015afcf345\" (UID: \"4586f536-8969-4d57-98c1-aa015afcf345\") "
Oct 30 22:48:31.842564 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:31.842530    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4586f536-8969-4d57-98c1-aa015afcf345-tls-certs" pod "4586f536-8969-4d57-98c1-aa015afcf345" (UID: "4586f536-8969-4d57-98c1-aa015afcf345"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:48:31.842564 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:31.842547    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4586f536-8969-4d57-98c1-aa015afcf345-kube-api-access-68mr7" pod "4586f536-8969-4d57-98c1-aa015afcf345" (UID: "4586f536-8969-4d57-98c1-aa015afcf345"). InnerVolumeSpecName "kube-api-access-68mr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:48:31.941127 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:31.941100    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68mr7\" (UniqueName: \"kubernetes.io/projected/4586f536-8969-4d57-98c1-aa015afcf345-kube-api-access-68mr7\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:31.941127 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:31.941127    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4586f536-8969-4d57-98c1-aa015afcf345-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:48:32.078338 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.078308    5503 generic.go:356] "Generic (PLEG): container finished" podID="4586f536-8969-4d57-98c1-aa015afcf345" containerID="9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0" exitCode=0
Oct 30 22:48:32.078493 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.078366    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"
Oct 30 22:48:32.078493 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.078403    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" event={"ID":"4586f536-8969-4d57-98c1-aa015afcf345","Type":"ContainerDied","Data":"9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0"}
Oct 30 22:48:32.078493 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.078462    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9" event={"ID":"4586f536-8969-4d57-98c1-aa015afcf345","Type":"ContainerDied","Data":"3e0cc81dbf911f33b2278447d93209f8f6913afc535b07898323d2a96e8849bf"}
Oct 30 22:48:32.078493 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.078488    5503 scope.go:122] "RemoveContainer" containerID="9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0"
Oct 30 22:48:32.088237 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.088221    5503 scope.go:122] "RemoveContainer" containerID="9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0"
Oct 30 22:48:32.088527 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:48:32.088504    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0\": container with ID starting with 9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0 not found: ID does not exist" containerID="9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0"
Oct 30 22:48:32.088603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.088541    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0"} err="failed to get container status \"9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0\": rpc error: code = NotFound desc = could not find container \"9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0\": container with ID starting with 9306182c1205e597fcce28e5f7b07e2670ec0f1b61ec3dfe931fd33f8269b2e0 not found: ID does not exist"
Oct 30 22:48:32.099388 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.099365    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"]
Oct 30 22:48:32.103978 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.103957    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-router-scheduler-5666w4cs9"]
Oct 30 22:48:32.128379 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.128362    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:32.224439 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:32.224417    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:33.855042 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:33.855010    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4586f536-8969-4d57-98c1-aa015afcf345" path="/var/lib/kubelet/pods/4586f536-8969-4d57-98c1-aa015afcf345/volumes"
Oct 30 22:48:36.812940 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:36.812913    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:36.960719 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:36.960693    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:37.065601 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:37.065532    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:38.076557 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.076516    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:38.345164 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.345133    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"]
Oct 30 22:48:38.345568 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.345555    5503 cpu_manager.go:414] "containerMap: removing container" podUID="4586f536-8969-4d57-98c1-aa015afcf345" containerName="main"
Oct 30 22:48:38.345611 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.345571    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4586f536-8969-4d57-98c1-aa015afcf345" containerName="main"
Oct 30 22:48:38.345611 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.345592    5503 cpu_manager.go:414] "containerMap: removing container" podUID="56767a23-a7b9-4418-8516-692469a7bcce" containerName="main"
Oct 30 22:48:38.345611 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.345597    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56767a23-a7b9-4418-8516-692469a7bcce" containerName="main"
Oct 30 22:48:38.345733 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.345655    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="56767a23-a7b9-4418-8516-692469a7bcce" containerName="main"
Oct 30 22:48:38.345733 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.345690    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="4586f536-8969-4d57-98c1-aa015afcf345" containerName="main"
Oct 30 22:48:38.348774 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.348755    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:48:38.353941 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.353918    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\""
Oct 30 22:48:38.354083 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.354036    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\""
Oct 30 22:48:38.354195 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.354177    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-5f72j\""
Oct 30 22:48:38.354844 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.354823    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-epp-sa-dockercfg-j5qbf\""
Oct 30 22:48:38.354928 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.354911    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\""
Oct 30 22:48:38.361262 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.361239    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"]
Oct 30 22:48:38.403620 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.403587    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:38.493834 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.493803    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgqtr\" (UniqueName: \"kubernetes.io/projected/ee356e73-b2a2-4c63-969c-365e545624b8-kube-api-access-vgqtr\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc\" (UID: \"ee356e73-b2a2-4c63-969c-365e545624b8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:48:38.493834 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.493835    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee356e73-b2a2-4c63-969c-365e545624b8-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc\" (UID: \"ee356e73-b2a2-4c63-969c-365e545624b8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:48:38.502012 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.501985    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:38.594547 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.594519    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgqtr\" (UniqueName: \"kubernetes.io/projected/ee356e73-b2a2-4c63-969c-365e545624b8-kube-api-access-vgqtr\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc\" (UID: \"ee356e73-b2a2-4c63-969c-365e545624b8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:48:38.594754 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.594553    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee356e73-b2a2-4c63-969c-365e545624b8-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc\" (UID: \"ee356e73-b2a2-4c63-969c-365e545624b8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:48:38.597364 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.597310    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee356e73-b2a2-4c63-969c-365e545624b8-tls-certs\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc\" (UID: \"ee356e73-b2a2-4c63-969c-365e545624b8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:48:38.609134 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.609106    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgqtr\" (UniqueName: \"kubernetes.io/projected/ee356e73-b2a2-4c63-969c-365e545624b8-kube-api-access-vgqtr\") pod \"tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc\" (UID: \"ee356e73-b2a2-4c63-969c-365e545624b8\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:48:38.666017 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.664889    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:48:38.954181 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:38.954153    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"]
Oct 30 22:48:38.956507 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:48:38.956470    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee356e73_b2a2_4c63_969c_365e545624b8.slice/crio-ec2a4250592d85f5ebb12e44df19cd2ad2458d52170ab16fb0710c16b6cba79d WatchSource:0}: Error finding container ec2a4250592d85f5ebb12e44df19cd2ad2458d52170ab16fb0710c16b6cba79d: Status 404 returned error can't find the container with id ec2a4250592d85f5ebb12e44df19cd2ad2458d52170ab16fb0710c16b6cba79d
Oct 30 22:48:39.120713 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:39.119187    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" event={"ID":"ee356e73-b2a2-4c63-969c-365e545624b8","Type":"ContainerStarted","Data":"a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a"}
Oct 30 22:48:39.120713 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:39.119253    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" event={"ID":"ee356e73-b2a2-4c63-969c-365e545624b8","Type":"ContainerStarted","Data":"ec2a4250592d85f5ebb12e44df19cd2ad2458d52170ab16fb0710c16b6cba79d"}
Oct 30 22:48:39.120713 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:39.120255    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:48:39.143693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:39.142652    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" podStartSLOduration=1.142631706 podStartE2EDuration="1.142631706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:48:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:48:39.142175555 +0000 UTC m=+1761.411257088" watchObservedRunningTime="2026-10-30 22:48:39.142631706 +0000 UTC m=+1761.411713431"
Oct 30 22:48:40.239693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:40.239116    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:43.026693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:43.026354    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:43.583696 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:43.583272    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:45.449362 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:48:45.447589    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:48:45.494208 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:45.494124    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:49.591619 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:49.591580    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:48:49.701858 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:48:49.701831    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:11.166247 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:11.166217    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:49:15.388557 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:49:15.388518    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:49:17.226779 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:17.226750    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"]
Oct 30 22:49:17.238278 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:17.238245    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:18.140650 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:18.140610    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:20.283589 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:20.283557    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:21.166404 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:21.166370    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:49:25.140831 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:25.140795    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:25.314637 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:25.314600    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:31.166412 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:31.166318    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:49:32.248715 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:32.248646    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" containerName="main" containerID="cri-o://a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a" gracePeriod=45
Oct 30 22:49:40.284578 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:40.284546    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:49:41.166178 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:41.166145    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:49:41.166352 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:41.166239    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:49:45.390780 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:49:45.390741    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:49:51.166801 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:49:51.166756    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:01.166148 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:01.166106    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:02.515874 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.515813    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:50:02.621465 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.621432    5503 generic.go:356] "Generic (PLEG): container finished" podID="ee356e73-b2a2-4c63-969c-365e545624b8" containerID="a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a" exitCode=0
Oct 30 22:50:02.621604 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.621480    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"
Oct 30 22:50:02.621604 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.621515    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" event={"ID":"ee356e73-b2a2-4c63-969c-365e545624b8","Type":"ContainerDied","Data":"a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a"}
Oct 30 22:50:02.621604 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.621568    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc" event={"ID":"ee356e73-b2a2-4c63-969c-365e545624b8","Type":"ContainerDied","Data":"ec2a4250592d85f5ebb12e44df19cd2ad2458d52170ab16fb0710c16b6cba79d"}
Oct 30 22:50:02.621604 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.621596    5503 scope.go:122] "RemoveContainer" containerID="a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a"
Oct 30 22:50:02.630448 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.630433    5503 scope.go:122] "RemoveContainer" containerID="a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a"
Oct 30 22:50:02.630716 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:50:02.630694    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a\": container with ID starting with a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a not found: ID does not exist" containerID="a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a"
Oct 30 22:50:02.630795 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.630725    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a"} err="failed to get container status \"a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a\": rpc error: code = NotFound desc = could not find container \"a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a\": container with ID starting with a7f103d9e655413a7b949b7df4aaaaea37eb0aefa5df1fe86efae27dcc7c031a not found: ID does not exist"
Oct 30 22:50:02.700703 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.700659    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee356e73-b2a2-4c63-969c-365e545624b8-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee356e73-b2a2-4c63-969c-365e545624b8-tls-certs\") pod \"ee356e73-b2a2-4c63-969c-365e545624b8\" (UID: \"ee356e73-b2a2-4c63-969c-365e545624b8\") "
Oct 30 22:50:02.700808 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.700796    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee356e73-b2a2-4c63-969c-365e545624b8-kube-api-access-vgqtr\" (UniqueName: \"kubernetes.io/projected/ee356e73-b2a2-4c63-969c-365e545624b8-kube-api-access-vgqtr\") pod \"ee356e73-b2a2-4c63-969c-365e545624b8\" (UID: \"ee356e73-b2a2-4c63-969c-365e545624b8\") "
Oct 30 22:50:02.702904 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.702882    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee356e73-b2a2-4c63-969c-365e545624b8-tls-certs" pod "ee356e73-b2a2-4c63-969c-365e545624b8" (UID: "ee356e73-b2a2-4c63-969c-365e545624b8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:50:02.702979 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.702931    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee356e73-b2a2-4c63-969c-365e545624b8-kube-api-access-vgqtr" pod "ee356e73-b2a2-4c63-969c-365e545624b8" (UID: "ee356e73-b2a2-4c63-969c-365e545624b8"). InnerVolumeSpecName "kube-api-access-vgqtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:50:02.802342 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.802294    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgqtr\" (UniqueName: \"kubernetes.io/projected/ee356e73-b2a2-4c63-969c-365e545624b8-kube-api-access-vgqtr\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.802342 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.802322    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee356e73-b2a2-4c63-969c-365e545624b8-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:50:02.947221 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.947191    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"]
Oct 30 22:50:02.952799 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:02.952776    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-router-scheduler-74bb62fjrc"]
Oct 30 22:50:03.855887 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:03.855855    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" path="/var/lib/kubelet/pods/ee356e73-b2a2-4c63-969c-365e545624b8/volumes"
Oct 30 22:50:07.501295 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:07.501261    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:07.625431 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:07.625397    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:07.751687 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:07.751607    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:09.585269 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.585240    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:09.796780 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.796745    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"]
Oct 30 22:50:09.797375 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.797354    5503 cpu_manager.go:414] "containerMap: removing container" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" containerName="main"
Oct 30 22:50:09.797488 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.797377    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" containerName="main"
Oct 30 22:50:09.797488 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.797476    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee356e73-b2a2-4c63-969c-365e545624b8" containerName="main"
Oct 30 22:50:09.801009 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.800985    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:09.806652 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.806621    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"kube-root-ca.crt\""
Oct 30 22:50:09.808719 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.808696    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"]
Oct 30 22:50:09.808825 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.808741    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"openshift-service-ca.crt\""
Oct 30 22:50:09.810097 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.810066    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs\""
Oct 30 22:50:09.810097 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.810074    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"llmisvc-router-managed-test-llm-e45d1f79-epp-sa-dockercfg-mwjgm\""
Oct 30 22:50:09.810277 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.810154    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-350756ca\"/\"default-dockercfg-8cj2k\""
Oct 30 22:50:09.853488 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.853464    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:09.864402 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.864381    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed44f741-e6e5-403a-b91a-ad4984716133-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm\" (UID: \"ed44f741-e6e5-403a-b91a-ad4984716133\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:09.864518 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.864443    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhwt4\" (UniqueName: \"kubernetes.io/projected/ed44f741-e6e5-403a-b91a-ad4984716133-kube-api-access-qhwt4\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm\" (UID: \"ed44f741-e6e5-403a-b91a-ad4984716133\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:09.965693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.965484    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed44f741-e6e5-403a-b91a-ad4984716133-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm\" (UID: \"ed44f741-e6e5-403a-b91a-ad4984716133\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:09.966116 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.966078    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhwt4\" (UniqueName: \"kubernetes.io/projected/ed44f741-e6e5-403a-b91a-ad4984716133-kube-api-access-qhwt4\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm\" (UID: \"ed44f741-e6e5-403a-b91a-ad4984716133\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:09.976942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.976915    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed44f741-e6e5-403a-b91a-ad4984716133-tls-certs\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm\" (UID: \"ed44f741-e6e5-403a-b91a-ad4984716133\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:09.981868 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:09.981846    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhwt4\" (UniqueName: \"kubernetes.io/projected/ed44f741-e6e5-403a-b91a-ad4984716133-kube-api-access-qhwt4\") pod \"llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm\" (UID: \"ed44f741-e6e5-403a-b91a-ad4984716133\") " pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:10.117725 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:10.117640    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:10.399889 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:10.399861    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"]
Oct 30 22:50:10.402626 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:50:10.402599    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded44f741_e6e5_403a_b91a_ad4984716133.slice/crio-8ec7274312fa953d71f507768f3e6c8d138e53d020b9aad2ce7ae0916718d38f WatchSource:0}: Error finding container 8ec7274312fa953d71f507768f3e6c8d138e53d020b9aad2ce7ae0916718d38f: Status 404 returned error can't find the container with id 8ec7274312fa953d71f507768f3e6c8d138e53d020b9aad2ce7ae0916718d38f
Oct 30 22:50:10.725752 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:10.723403    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" event={"ID":"ed44f741-e6e5-403a-b91a-ad4984716133","Type":"ContainerStarted","Data":"206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5"}
Oct 30 22:50:10.725752 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:10.723496    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" event={"ID":"ed44f741-e6e5-403a-b91a-ad4984716133","Type":"ContainerStarted","Data":"8ec7274312fa953d71f507768f3e6c8d138e53d020b9aad2ce7ae0916718d38f"}
Oct 30 22:50:10.729701 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:10.728697    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:10.768692 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:10.768129    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" podStartSLOduration=1.768110911 podStartE2EDuration="1.768110911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:50:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:50:10.76769582 +0000 UTC m=+1853.036777344" watchObservedRunningTime="2026-10-30 22:50:10.768110911 +0000 UTC m=+1853.037192432"
Oct 30 22:50:13.945710 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:13.945508    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:15.511688 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:50:15.507942    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:50:19.251010 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:19.250945    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:20.114750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:20.114711    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:20.216130 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:20.216102    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:20.344365 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:20.344332    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:25.196695 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:25.196630    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:34.551033 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:34.550986    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:34.768645 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:34.768597    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:40.338195 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:40.338134    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:40.438970 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:40.438942    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:42.812923 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:42.812897    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:50:44.720140 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:44.720103    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"]
Oct 30 22:50:44.745957 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:44.745928    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:44.850030 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:44.849995    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:50:45.389658 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:50:45.389624    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:50:52.813165 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:52.813121    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" podUID="ed44f741-e6e5-403a-b91a-ad4984716133" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:50:59.742200 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:50:59.742157    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" podUID="ed44f741-e6e5-403a-b91a-ad4984716133" containerName="main" containerID="cri-o://206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5" gracePeriod=45
Oct 30 22:51:02.813111 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:02.813068    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" podUID="ed44f741-e6e5-403a-b91a-ad4984716133" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:51:12.812823 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:12.812776    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" podUID="ed44f741-e6e5-403a-b91a-ad4984716133" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:51:12.813260 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:12.812916    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:51:15.392858 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:51:15.392805    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:51:22.813070 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:22.813028    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" podUID="ed44f741-e6e5-403a-b91a-ad4984716133" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:51:25.261249 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:25.261210    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:29.995871 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:29.995846    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:51:30.085292 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.085235    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ed44f741-e6e5-403a-b91a-ad4984716133-kube-api-access-qhwt4\" (UniqueName: \"kubernetes.io/projected/ed44f741-e6e5-403a-b91a-ad4984716133-kube-api-access-qhwt4\") pod \"ed44f741-e6e5-403a-b91a-ad4984716133\" (UID: \"ed44f741-e6e5-403a-b91a-ad4984716133\") "
Oct 30 22:51:30.085292 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.085291    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ed44f741-e6e5-403a-b91a-ad4984716133-tls-certs\" (UniqueName: \"kubernetes.io/secret/ed44f741-e6e5-403a-b91a-ad4984716133-tls-certs\") pod \"ed44f741-e6e5-403a-b91a-ad4984716133\" (UID: \"ed44f741-e6e5-403a-b91a-ad4984716133\") "
Oct 30 22:51:30.087512 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.087485    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed44f741-e6e5-403a-b91a-ad4984716133-tls-certs" pod "ed44f741-e6e5-403a-b91a-ad4984716133" (UID: "ed44f741-e6e5-403a-b91a-ad4984716133"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:51:30.087594 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.087527    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed44f741-e6e5-403a-b91a-ad4984716133-kube-api-access-qhwt4" pod "ed44f741-e6e5-403a-b91a-ad4984716133" (UID: "ed44f741-e6e5-403a-b91a-ad4984716133"). InnerVolumeSpecName "kube-api-access-qhwt4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:51:30.186367 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.186333    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhwt4\" (UniqueName: \"kubernetes.io/projected/ed44f741-e6e5-403a-b91a-ad4984716133-kube-api-access-qhwt4\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:30.186367 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.186366    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed44f741-e6e5-403a-b91a-ad4984716133-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:51:30.261751 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.261723    5503 generic.go:356] "Generic (PLEG): container finished" podID="ed44f741-e6e5-403a-b91a-ad4984716133" containerID="206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5" exitCode=0
Oct 30 22:51:30.261848 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.261756    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" event={"ID":"ed44f741-e6e5-403a-b91a-ad4984716133","Type":"ContainerDied","Data":"206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5"}
Oct 30 22:51:30.261848 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.261787    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"
Oct 30 22:51:30.261848 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.261812    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm" event={"ID":"ed44f741-e6e5-403a-b91a-ad4984716133","Type":"ContainerDied","Data":"8ec7274312fa953d71f507768f3e6c8d138e53d020b9aad2ce7ae0916718d38f"}
Oct 30 22:51:30.261848 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.261837    5503 scope.go:122] "RemoveContainer" containerID="206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5"
Oct 30 22:51:30.271502 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.271484    5503 scope.go:122] "RemoveContainer" containerID="206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5"
Oct 30 22:51:30.271821 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:51:30.271800    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5\": container with ID starting with 206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5 not found: ID does not exist" containerID="206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5"
Oct 30 22:51:30.271888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.271834    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5"} err="failed to get container status \"206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5\": rpc error: code = NotFound desc = could not find container \"206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5\": container with ID starting with 206fa8c8f6f643cbf3af6769dd016480bb7e714229cea7a576f4f94347f216f5 not found: ID does not exist"
Oct 30 22:51:30.285298 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.285273    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"]
Oct 30 22:51:30.290153 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:30.290116    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-350756ca/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheqfbrm"]
Oct 30 22:51:31.854882 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:31.854848    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ed44f741-e6e5-403a-b91a-ad4984716133" path="/var/lib/kubelet/pods/ed44f741-e6e5-403a-b91a-ad4984716133/volumes"
Oct 30 22:51:34.926300 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:34.926268    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:35.034899 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:35.034870    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:40.392500 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.392465    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:40.790979 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.790907    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"]
Oct 30 22:51:40.791309 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.791283    5503 cpu_manager.go:414] "containerMap: removing container" podUID="ed44f741-e6e5-403a-b91a-ad4984716133" containerName="main"
Oct 30 22:51:40.791309 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.791303    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ed44f741-e6e5-403a-b91a-ad4984716133" containerName="main"
Oct 30 22:51:40.791574 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.791458    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="ed44f741-e6e5-403a-b91a-ad4984716133" containerName="main"
Oct 30 22:51:40.794652 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.794633    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.799901 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.799880    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"kube-root-ca.crt\""
Oct 30 22:51:40.800021 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.800007    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"openshift-service-ca.crt\""
Oct 30 22:51:40.800094 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.800010    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"default-dockercfg-jgwjb\""
Oct 30 22:51:40.800925 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.800906    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs\""
Oct 30 22:51:40.806555 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.806534    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"]
Oct 30 22:51:40.834424 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.834398    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:40.878936 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.878906    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7bgf\" (UniqueName: \"kubernetes.io/projected/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-kube-api-access-g7bgf\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.879041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.878950    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.879041 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.879021    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.879130 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.879065    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.879130 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.879109    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.879237 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.879132    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.980160 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.980130    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.980268 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.980170    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.980268 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.980239    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7bgf\" (UniqueName: \"kubernetes.io/projected/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-kube-api-access-g7bgf\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.980389 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.980288    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.980389 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.980325    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.980389 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.980365    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.980600 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.980579    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tmp-dir\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.980728 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.980708    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-home\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.980829 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.980809    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-model-cache\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.982721 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.982703    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-dshm\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.983020 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.983001    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tls-certs\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:40.993425 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:40.993404    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7bgf\" (UniqueName: \"kubernetes.io/projected/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-kube-api-access-g7bgf\") pod \"llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") " pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:41.063881 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.063802    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"]
Oct 30 22:51:41.068295 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.068267    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:51:41.080171 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.080143    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"]
Oct 30 22:51:41.100682 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.100643    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvc-router-managed-test-llm-5b1e8f15-epp-sa-dockercfg-kzvb5\""
Oct 30 22:51:41.107891 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.107866    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:41.123993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.123965    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:41.182080 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.181909    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btjw9\" (UniqueName: \"kubernetes.io/projected/b545713d-f41c-491b-a38e-42c2a82c8bd7-kube-api-access-btjw9\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798\" (UID: \"b545713d-f41c-491b-a38e-42c2a82c8bd7\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:51:41.182080 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.181978    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b545713d-f41c-491b-a38e-42c2a82c8bd7-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798\" (UID: \"b545713d-f41c-491b-a38e-42c2a82c8bd7\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:51:41.283067 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.283032    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b545713d-f41c-491b-a38e-42c2a82c8bd7-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798\" (UID: \"b545713d-f41c-491b-a38e-42c2a82c8bd7\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:51:41.283239 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.283120    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btjw9\" (UniqueName: \"kubernetes.io/projected/b545713d-f41c-491b-a38e-42c2a82c8bd7-kube-api-access-btjw9\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798\" (UID: \"b545713d-f41c-491b-a38e-42c2a82c8bd7\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:51:41.285795 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.285776    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b545713d-f41c-491b-a38e-42c2a82c8bd7-tls-certs\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798\" (UID: \"b545713d-f41c-491b-a38e-42c2a82c8bd7\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:51:41.296358 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.296329    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btjw9\" (UniqueName: \"kubernetes.io/projected/b545713d-f41c-491b-a38e-42c2a82c8bd7-kube-api-access-btjw9\") pod \"llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798\" (UID: \"b545713d-f41c-491b-a38e-42c2a82c8bd7\") " pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:51:41.384150 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.384126    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:51:41.390019 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.389999    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"]
Oct 30 22:51:41.394252 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:51:41.394220    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7a63ec9_6a8b_45dc_9f7b_62ce563801e1.slice/crio-d9d65dc81da323504d15549ec9e14093271e49ec8403e500dadc56ad26142b01 WatchSource:0}: Error finding container d9d65dc81da323504d15549ec9e14093271e49ec8403e500dadc56ad26142b01: Status 404 returned error can't find the container with id d9d65dc81da323504d15549ec9e14093271e49ec8403e500dadc56ad26142b01
Oct 30 22:51:41.682829 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.682794    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"]
Oct 30 22:51:41.684823 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:51:41.684779    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb545713d_f41c_491b_a38e_42c2a82c8bd7.slice/crio-0ccf961fd1e5fac925303104f8f557ee0cbc6c0ce89066a746b7d73411a34d9f WatchSource:0}: Error finding container 0ccf961fd1e5fac925303104f8f557ee0cbc6c0ce89066a746b7d73411a34d9f: Status 404 returned error can't find the container with id 0ccf961fd1e5fac925303104f8f557ee0cbc6c0ce89066a746b7d73411a34d9f
Oct 30 22:51:41.981740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:41.977142    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:42.324997 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:42.324904    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" event={"ID":"b545713d-f41c-491b-a38e-42c2a82c8bd7","Type":"ContainerStarted","Data":"403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5"}
Oct 30 22:51:42.324997 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:42.324977    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:51:42.325221 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:42.325000    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" event={"ID":"b545713d-f41c-491b-a38e-42c2a82c8bd7","Type":"ContainerStarted","Data":"0ccf961fd1e5fac925303104f8f557ee0cbc6c0ce89066a746b7d73411a34d9f"}
Oct 30 22:51:42.326806 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:42.326775    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm" event={"ID":"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1","Type":"ContainerStarted","Data":"8a4bc52681c42080499ae19ac9e7cd763d2be95fe604a2897440ac1c14005dfe"}
Oct 30 22:51:42.326956 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:42.326826    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm" event={"ID":"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1","Type":"ContainerStarted","Data":"d9d65dc81da323504d15549ec9e14093271e49ec8403e500dadc56ad26142b01"}
Oct 30 22:51:42.346209 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:42.346166    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" podStartSLOduration=1.346153239 podStartE2EDuration="1.346153239s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:51:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:51:42.344719894 +0000 UTC m=+1944.613801420" watchObservedRunningTime="2026-10-30 22:51:42.346153239 +0000 UTC m=+1944.615234811"
Oct 30 22:51:42.367614 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:42.367575    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm" podStartSLOduration=2.367564098 podStartE2EDuration="2.367564098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:51:42.365970667 +0000 UTC m=+1944.635052192" watchObservedRunningTime="2026-10-30 22:51:42.367564098 +0000 UTC m=+1944.636645621"
Oct 30 22:51:43.108655 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:43.108250    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:45.389270 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:51:45.389236    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:51:47.680926 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:47.680883    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:47.784627 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:47.784598    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:48.450786 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:48.450753    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:51:51.108742 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:51.108706    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:51.121462 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:51.121436    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:51:51.392850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:51:51.392763    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:52:13.344029 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:13.343997    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:52:15.390848 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:52:15.390800    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:52:16.606947 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:16.606914    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:16.815951 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:16.815914    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"]
Oct 30 22:52:16.820627 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:16.820595    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:16.834090 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:16.834066    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvc-model-qwen2-5-0-5b-rout-a50492e9-epp-sa-dockercfg-vcxb4\""
Oct 30 22:52:16.834369 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:16.834345    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-84370a53\"/\"llmisvb11a9c9832b99b016bc8f8e0ea095712-kserve-self-signed-certs\""
Oct 30 22:52:16.836387 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:16.836357    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"]
Oct 30 22:52:16.915991 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:16.915917    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b430a09-659c-4d49-b92b-8519ee3b3e4a-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx\" (UID: \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:16.916142 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:16.916007    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wclxz\" (UniqueName: \"kubernetes.io/projected/8b430a09-659c-4d49-b92b-8519ee3b3e4a-kube-api-access-wclxz\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx\" (UID: \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:16.929239 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:16.929208    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:17.017336 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.017304    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b430a09-659c-4d49-b92b-8519ee3b3e4a-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx\" (UID: \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:17.017515 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.017375    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wclxz\" (UniqueName: \"kubernetes.io/projected/8b430a09-659c-4d49-b92b-8519ee3b3e4a-kube-api-access-wclxz\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx\" (UID: \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:17.020289 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.020257    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b430a09-659c-4d49-b92b-8519ee3b3e4a-tls-certs\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx\" (UID: \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:17.031382 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.031353    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wclxz\" (UniqueName: \"kubernetes.io/projected/8b430a09-659c-4d49-b92b-8519ee3b3e4a-kube-api-access-wclxz\") pod \"llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx\" (UID: \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\") " pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:17.135331 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.135296    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:17.436565 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.436483    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"]
Oct 30 22:52:17.438216 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:52:17.438189    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b430a09_659c_4d49_b92b_8519ee3b3e4a.slice/crio-f849ff92a5777af6922606414d6f554b73ede4181868ca369236f2f8c3526dec WatchSource:0}: Error finding container f849ff92a5777af6922606414d6f554b73ede4181868ca369236f2f8c3526dec: Status 404 returned error can't find the container with id f849ff92a5777af6922606414d6f554b73ede4181868ca369236f2f8c3526dec
Oct 30 22:52:17.531729 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.531699    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" event={"ID":"8b430a09-659c-4d49-b92b-8519ee3b3e4a","Type":"ContainerStarted","Data":"ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca"}
Oct 30 22:52:17.531903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.531762    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" event={"ID":"8b430a09-659c-4d49-b92b-8519ee3b3e4a","Type":"ContainerStarted","Data":"f849ff92a5777af6922606414d6f554b73ede4181868ca369236f2f8c3526dec"}
Oct 30 22:52:17.531903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.531791    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:17.582803 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:17.582753    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" podStartSLOduration=1.582739112 podStartE2EDuration="1.582739112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:52:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:52:17.581852 +0000 UTC m=+1979.850933524" watchObservedRunningTime="2026-10-30 22:52:17.582739112 +0000 UTC m=+1979.851820635"
Oct 30 22:52:25.305756 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:25.305708    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:28.514300 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:28.514212    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:40.453349 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:40.453312    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:45.390329 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:52:45.390294    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:52:47.213514 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:47.213479    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:47.363646 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:47.363609    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:48.538639 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:48.538605    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:52:50.091286 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:50.091249    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"]
Oct 30 22:52:50.122145 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:50.122112    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:50.245612 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:50.245576    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:57.036922 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:57.036892    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:57.189551 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:57.189523    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:52:58.538460 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:52:58.538420    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" podUID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:53:01.258470 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:01.258436    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:01.973819 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:01.973788    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:05.112910 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:05.112873    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" podUID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" containerName="main" containerID="cri-o://ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca" gracePeriod=45
Oct 30 22:53:08.538394 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:08.538356    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" podUID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:53:11.453255 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:11.453227    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:11.592029 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:11.592004    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:15.388249 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:53:15.388215    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:53:18.538796 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:18.538763    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" podUID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:53:18.539278 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:18.538854    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:53:25.436005 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:25.435970    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:28.538269 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:28.538228    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" podUID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:53:35.364876 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.364852    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:53:35.500088 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.500023    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8b430a09-659c-4d49-b92b-8519ee3b3e4a-tls-certs\" (UniqueName: \"kubernetes.io/secret/8b430a09-659c-4d49-b92b-8519ee3b3e4a-tls-certs\") pod \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\" (UID: \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\") "
Oct 30 22:53:35.500223 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.500092    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b430a09-659c-4d49-b92b-8519ee3b3e4a-kube-api-access-wclxz\" (UniqueName: \"kubernetes.io/projected/8b430a09-659c-4d49-b92b-8519ee3b3e4a-kube-api-access-wclxz\") pod \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\" (UID: \"8b430a09-659c-4d49-b92b-8519ee3b3e4a\") "
Oct 30 22:53:35.502236 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.502212    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b430a09-659c-4d49-b92b-8519ee3b3e4a-kube-api-access-wclxz" pod "8b430a09-659c-4d49-b92b-8519ee3b3e4a" (UID: "8b430a09-659c-4d49-b92b-8519ee3b3e4a"). InnerVolumeSpecName "kube-api-access-wclxz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:35.502236 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.502225    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b430a09-659c-4d49-b92b-8519ee3b3e4a-tls-certs" pod "8b430a09-659c-4d49-b92b-8519ee3b3e4a" (UID: "8b430a09-659c-4d49-b92b-8519ee3b3e4a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:53:35.601321 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.601300    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8b430a09-659c-4d49-b92b-8519ee3b3e4a-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:35.601413 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.601323    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wclxz\" (UniqueName: \"kubernetes.io/projected/8b430a09-659c-4d49-b92b-8519ee3b3e4a-kube-api-access-wclxz\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:35.948612 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.948584    5503 generic.go:356] "Generic (PLEG): container finished" podID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" containerID="ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca" exitCode=0
Oct 30 22:53:35.948764 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.948627    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" event={"ID":"8b430a09-659c-4d49-b92b-8519ee3b3e4a","Type":"ContainerDied","Data":"ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca"}
Oct 30 22:53:35.948764 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.948633    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"
Oct 30 22:53:35.948764 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.948658    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx" event={"ID":"8b430a09-659c-4d49-b92b-8519ee3b3e4a","Type":"ContainerDied","Data":"f849ff92a5777af6922606414d6f554b73ede4181868ca369236f2f8c3526dec"}
Oct 30 22:53:35.948764 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.948704    5503 scope.go:122] "RemoveContainer" containerID="ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca"
Oct 30 22:53:35.964003 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.963984    5503 scope.go:122] "RemoveContainer" containerID="ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca"
Oct 30 22:53:35.964275 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:53:35.964255    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca\": container with ID starting with ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca not found: ID does not exist" containerID="ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca"
Oct 30 22:53:35.964324 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.964283    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca"} err="failed to get container status \"ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca\": rpc error: code = NotFound desc = could not find container \"ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca\": container with ID starting with ecd9fe0b6c3747d977e5adc7102ea089832bc1fb00870b0413063e372ff53dca not found: ID does not exist"
Oct 30 22:53:35.972238 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.972210    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"]
Oct 30 22:53:35.974935 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:35.974914    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvcb11a9c9832b99b016bc8f8e0ea095712-kserve-router-schemflbx"]
Oct 30 22:53:37.856546 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:37.856514    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" path="/var/lib/kubelet/pods/8b430a09-659c-4d49-b92b-8519ee3b3e4a/volumes"
Oct 30 22:53:40.183014 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.182982    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:40.480256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.480183    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"]
Oct 30 22:53:40.487781 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.487754    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"]
Oct 30 22:53:40.490591 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:53:40.490553    5503 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="8a4bc52681c42080499ae19ac9e7cd763d2be95fe604a2897440ac1c14005dfe" cmd=["/bin/sleep","15"]
Oct 30 22:53:40.490767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.490615    5503 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:53:40.490767 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:53:40.490633    5503 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm" podUID="b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" containerName="main" containerID="cri-o://8a4bc52681c42080499ae19ac9e7cd763d2be95fe604a2897440ac1c14005dfe"
Oct 30 22:53:40.490767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.490689    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm" podUID="b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" containerName="main" containerID="cri-o://8a4bc52681c42080499ae19ac9e7cd763d2be95fe604a2897440ac1c14005dfe" gracePeriod=60
Oct 30 22:53:40.513053 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.513029    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:40.618036 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.618006    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:40.881852 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.881820    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:40.981085 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.981059    5503 generic.go:356] "Generic (PLEG): container finished" podID="b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" containerID="8a4bc52681c42080499ae19ac9e7cd763d2be95fe604a2897440ac1c14005dfe" exitCode=0
Oct 30 22:53:40.981237 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:40.981147    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm" event={"ID":"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1","Type":"ContainerDied","Data":"8a4bc52681c42080499ae19ac9e7cd763d2be95fe604a2897440ac1c14005dfe"}
Oct 30 22:53:41.240213 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.240190    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:53:41.348406 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.348378    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tls-certs\" (UniqueName: \"kubernetes.io/secret/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tls-certs\") pod \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") "
Oct 30 22:53:41.348526 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.348417    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tmp-dir\") pod \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") "
Oct 30 22:53:41.348526 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.348466    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-kube-api-access-g7bgf\" (UniqueName: \"kubernetes.io/projected/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-kube-api-access-g7bgf\") pod \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") "
Oct 30 22:53:41.348648 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.348561    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-dshm\") pod \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") "
Oct 30 22:53:41.348648 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.348623    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-home\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-home\") pod \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") "
Oct 30 22:53:41.348791 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.348712    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-model-cache\") pod \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\" (UID: \"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1\") "
Oct 30 22:53:41.348791 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.348749    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tmp-dir" pod "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" (UID: "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:41.348919 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.348890    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-home" pod "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" (UID: "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:41.349028 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.348996    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-model-cache" pod "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" (UID: "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:41.349209 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.349079    5503 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-home\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:41.349209 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.349100    5503 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-model-cache\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:41.349209 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.349138    5503 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tmp-dir\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:41.350718 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.350695    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-dshm" pod "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" (UID: "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:41.350794 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.350726    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tls-certs" pod "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" (UID: "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:53:41.350836 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.350791    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-kube-api-access-g7bgf" pod "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" (UID: "b7a63ec9-6a8b-45dc-9f7b-62ce563801e1"). InnerVolumeSpecName "kube-api-access-g7bgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:41.450038 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.449988    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:41.450038 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.450009    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7bgf\" (UniqueName: \"kubernetes.io/projected/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-kube-api-access-g7bgf\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:41.450038 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.450019    5503 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1-dshm\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:41.987500 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.987478    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"
Oct 30 22:53:41.987500 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.987487    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm" event={"ID":"b7a63ec9-6a8b-45dc-9f7b-62ce563801e1","Type":"ContainerDied","Data":"d9d65dc81da323504d15549ec9e14093271e49ec8403e500dadc56ad26142b01"}
Oct 30 22:53:41.987693 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:41.987541    5503 scope.go:122] "RemoveContainer" containerID="8a4bc52681c42080499ae19ac9e7cd763d2be95fe604a2897440ac1c14005dfe"
Oct 30 22:53:42.006943 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:42.006921    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"]
Oct 30 22:53:42.011841 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:42.011821    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvc-router-managed-test-llm-5b1e8f15-kserve-5957d55848qqrrm"]
Oct 30 22:53:43.344313 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:43.344283    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:53:43.855422 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:43.855382    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" path="/var/lib/kubelet/pods/b7a63ec9-6a8b-45dc-9f7b-62ce563801e1/volumes"
Oct 30 22:53:45.389093 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:53:45.389059    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:53:46.545327 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:46.545300    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:46.654158 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:46.654128    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:46.757052 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:46.757018    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:50.606640 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:50.606597    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:50.869457 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:50.869331    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:52.103210 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:52.103169    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:53.155391 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:53.155343    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:53.343836 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:53.343796    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:53:55.510428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:55.510389    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerName="main" containerID="cri-o://403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5" gracePeriod=45
Oct 30 22:53:59.269819 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:59.269787    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:53:59.372759 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:53:59.372732    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:03.343446 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:03.343412    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:54:03.343895 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:03.343516    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:54:13.344381 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:13.344344    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:54:15.389200 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:54:15.389165    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:54:23.344288 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:23.344247    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:54:25.457790 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:25.457754    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:25.763899 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:25.763876    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:54:25.913894 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:25.913870    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b545713d-f41c-491b-a38e-42c2a82c8bd7-tls-certs\" (UniqueName: \"kubernetes.io/secret/b545713d-f41c-491b-a38e-42c2a82c8bd7-tls-certs\") pod \"b545713d-f41c-491b-a38e-42c2a82c8bd7\" (UID: \"b545713d-f41c-491b-a38e-42c2a82c8bd7\") "
Oct 30 22:54:25.914026 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:25.913951    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b545713d-f41c-491b-a38e-42c2a82c8bd7-kube-api-access-btjw9\" (UniqueName: \"kubernetes.io/projected/b545713d-f41c-491b-a38e-42c2a82c8bd7-kube-api-access-btjw9\") pod \"b545713d-f41c-491b-a38e-42c2a82c8bd7\" (UID: \"b545713d-f41c-491b-a38e-42c2a82c8bd7\") "
Oct 30 22:54:25.916060 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:25.916027    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b545713d-f41c-491b-a38e-42c2a82c8bd7-tls-certs" pod "b545713d-f41c-491b-a38e-42c2a82c8bd7" (UID: "b545713d-f41c-491b-a38e-42c2a82c8bd7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:54:25.916166 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:25.916092    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b545713d-f41c-491b-a38e-42c2a82c8bd7-kube-api-access-btjw9" pod "b545713d-f41c-491b-a38e-42c2a82c8bd7" (UID: "b545713d-f41c-491b-a38e-42c2a82c8bd7"). InnerVolumeSpecName "kube-api-access-btjw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:54:26.015181 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.015126    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b545713d-f41c-491b-a38e-42c2a82c8bd7-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:54:26.015181 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.015149    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btjw9\" (UniqueName: \"kubernetes.io/projected/b545713d-f41c-491b-a38e-42c2a82c8bd7-kube-api-access-btjw9\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:54:26.198830 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.198798    5503 generic.go:356] "Generic (PLEG): container finished" podID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerID="403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5" exitCode=0
Oct 30 22:54:26.198938 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.198840    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"
Oct 30 22:54:26.198938 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.198885    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" event={"ID":"b545713d-f41c-491b-a38e-42c2a82c8bd7","Type":"ContainerDied","Data":"403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5"}
Oct 30 22:54:26.198938 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.198928    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798" event={"ID":"b545713d-f41c-491b-a38e-42c2a82c8bd7","Type":"ContainerDied","Data":"0ccf961fd1e5fac925303104f8f557ee0cbc6c0ce89066a746b7d73411a34d9f"}
Oct 30 22:54:26.199065 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.198949    5503 scope.go:122] "RemoveContainer" containerID="403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5"
Oct 30 22:54:26.209005 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.208874    5503 scope.go:122] "RemoveContainer" containerID="403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5"
Oct 30 22:54:26.213179 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:54:26.213151    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5\": container with ID starting with 403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5 not found: ID does not exist" containerID="403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5"
Oct 30 22:54:26.213264 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.213187    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5"} err="failed to get container status \"403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5\": rpc error: code = NotFound desc = could not find container \"403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5\": container with ID starting with 403ef2559e84b820ca6df20567fa76ecd1dc835f8e5b36922f00f939ca37c0f5 not found: ID does not exist"
Oct 30 22:54:26.222773 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.222749    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"]
Oct 30 22:54:26.226958 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:26.226938    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-84370a53/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche2l798"]
Oct 30 22:54:27.855183 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:27.855153    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" path="/var/lib/kubelet/pods/b545713d-f41c-491b-a38e-42c2a82c8bd7/volumes"
Oct 30 22:54:30.725579 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:30.725550    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:30.881983 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:30.881955    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:34.440624 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.440593    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:34.616097 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616065    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"]
Oct 30 22:54:34.616450 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616438    5503 cpu_manager.go:414] "containerMap: removing container" podUID="b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" containerName="main"
Oct 30 22:54:34.616499 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616451    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" containerName="main"
Oct 30 22:54:34.616499 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616462    5503 cpu_manager.go:414] "containerMap: removing container" podUID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" containerName="main"
Oct 30 22:54:34.616499 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616466    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" containerName="main"
Oct 30 22:54:34.616499 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616485    5503 cpu_manager.go:414] "containerMap: removing container" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerName="main"
Oct 30 22:54:34.616499 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616491    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerName="main"
Oct 30 22:54:34.616646 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616558    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="b545713d-f41c-491b-a38e-42c2a82c8bd7" containerName="main"
Oct 30 22:54:34.616646 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616570    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7a63ec9-6a8b-45dc-9f7b-62ce563801e1" containerName="main"
Oct 30 22:54:34.616646 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.616575    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b430a09-659c-4d49-b92b-8519ee3b3e4a" containerName="main"
Oct 30 22:54:34.621527 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.621503    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:54:34.630946 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.630923    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Oct 30 22:54:34.631061 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.630954    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Oct 30 22:54:34.631820 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.631799    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-6zd5k\""
Oct 30 22:54:34.631948 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.631856    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvc-router-managed-test-llm-4b931143-epp-sa-dockercfg-gfj56\""
Oct 30 22:54:34.632016 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.631955    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\""
Oct 30 22:54:34.636263 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.636238    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"]
Oct 30 22:54:34.672370 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.672333    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:34.782526 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.782453    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30c69508-e130-413a-b829-524a9cafb973-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m\" (UID: \"30c69508-e130-413a-b829-524a9cafb973\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:54:34.782526 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.782487    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcd5p\" (UniqueName: \"kubernetes.io/projected/30c69508-e130-413a-b829-524a9cafb973-kube-api-access-fcd5p\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m\" (UID: \"30c69508-e130-413a-b829-524a9cafb973\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:54:34.883993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.883961    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30c69508-e130-413a-b829-524a9cafb973-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m\" (UID: \"30c69508-e130-413a-b829-524a9cafb973\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:54:34.883993 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.883994    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcd5p\" (UniqueName: \"kubernetes.io/projected/30c69508-e130-413a-b829-524a9cafb973-kube-api-access-fcd5p\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m\" (UID: \"30c69508-e130-413a-b829-524a9cafb973\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:54:34.886738 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.886714    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30c69508-e130-413a-b829-524a9cafb973-tls-certs\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m\" (UID: \"30c69508-e130-413a-b829-524a9cafb973\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:54:34.898461 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.898435    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcd5p\" (UniqueName: \"kubernetes.io/projected/30c69508-e130-413a-b829-524a9cafb973-kube-api-access-fcd5p\") pod \"llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m\" (UID: \"30c69508-e130-413a-b829-524a9cafb973\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:54:34.940432 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:34.940398    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:54:35.242344 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:35.242321    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"]
Oct 30 22:54:35.244880 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:54:35.244852    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30c69508_e130_413a_b829_524a9cafb973.slice/crio-63915edd23b40a240190701b252857831e1bbbbcdacecc8ef1d9c6ff12c587c4 WatchSource:0}: Error finding container 63915edd23b40a240190701b252857831e1bbbbcdacecc8ef1d9c6ff12c587c4: Status 404 returned error can't find the container with id 63915edd23b40a240190701b252857831e1bbbbcdacecc8ef1d9c6ff12c587c4
Oct 30 22:54:36.251102 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:36.251070    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" event={"ID":"30c69508-e130-413a-b829-524a9cafb973","Type":"ContainerStarted","Data":"ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8"}
Oct 30 22:54:36.251574 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:36.251122    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" event={"ID":"30c69508-e130-413a-b829-524a9cafb973","Type":"ContainerStarted","Data":"63915edd23b40a240190701b252857831e1bbbbcdacecc8ef1d9c6ff12c587c4"}
Oct 30 22:54:36.251574 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:36.251197    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:54:36.276255 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:36.276210    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" podStartSLOduration=2.276200075 podStartE2EDuration="2.276200075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:54:36.274720278 +0000 UTC m=+2118.543801801" watchObservedRunningTime="2026-10-30 22:54:36.276200075 +0000 UTC m=+2118.545281598"
Oct 30 22:54:37.430934 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:37.430884    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:37.750601 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:37.750516    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:40.567751 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:40.567700    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:43.160862 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:43.160826    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:43.265081 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:54:43.265055    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:54:45.389080 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:54:45.389049    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:55:07.257512 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:07.257482    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:55:12.258864 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.258836    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"]
Oct 30 22:55:12.262380 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.262360    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.281904 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.281876    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.282025 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.281919    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.282176 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.282156    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/286dcd01-9773-4e55-a391-f22a292e5141-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.282255 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.282214    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.282317 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.282309    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.282366 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.282354    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rqzp\" (UniqueName: \"kubernetes.io/projected/286dcd01-9773-4e55-a391-f22a292e5141-kube-api-access-2rqzp\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.284428 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.284409    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Oct 30 22:55:12.286547 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.286527    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"]
Oct 30 22:55:12.311782 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.311761    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:12.384660 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.384624    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.384921 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.384726    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.384921 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.384786    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rqzp\" (UniqueName: \"kubernetes.io/projected/286dcd01-9773-4e55-a391-f22a292e5141-kube-api-access-2rqzp\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.384921 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.384833    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.389281 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.389218    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.390574 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.390551    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/286dcd01-9773-4e55-a391-f22a292e5141-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.391287 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.391262    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.391395 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.391349    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.391496 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.391478    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.393649 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.393627    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.393750 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.393716    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/286dcd01-9773-4e55-a391-f22a292e5141-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.404621 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.404598    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rqzp\" (UniqueName: \"kubernetes.io/projected/286dcd01-9773-4e55-a391-f22a292e5141-kube-api-access-2rqzp\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.511873 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.511790    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"]
Oct 30 22:55:12.516302 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.516281    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:12.529823 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.529794    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-epp-sa-dockercfg-njgzg\""
Oct 30 22:55:12.536316 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.536291    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"]
Oct 30 22:55:12.575219 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.575196    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:12.593380 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.593360    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60baf000-5ece-459d-85f8-18e11e72217e-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx\" (UID: \"60baf000-5ece-459d-85f8-18e11e72217e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:12.593468 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.593400    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z7bb\" (UniqueName: \"kubernetes.io/projected/60baf000-5ece-459d-85f8-18e11e72217e-kube-api-access-9z7bb\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx\" (UID: \"60baf000-5ece-459d-85f8-18e11e72217e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:12.596861 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.596837    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:12.694471 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.694445    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z7bb\" (UniqueName: \"kubernetes.io/projected/60baf000-5ece-459d-85f8-18e11e72217e-kube-api-access-9z7bb\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx\" (UID: \"60baf000-5ece-459d-85f8-18e11e72217e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:12.694609 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.694548    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60baf000-5ece-459d-85f8-18e11e72217e-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx\" (UID: \"60baf000-5ece-459d-85f8-18e11e72217e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:12.697117 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.697085    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60baf000-5ece-459d-85f8-18e11e72217e-tls-certs\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx\" (UID: \"60baf000-5ece-459d-85f8-18e11e72217e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:12.711771 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.711747    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z7bb\" (UniqueName: \"kubernetes.io/projected/60baf000-5ece-459d-85f8-18e11e72217e-kube-api-access-9z7bb\") pod \"llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx\" (UID: \"60baf000-5ece-459d-85f8-18e11e72217e\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:12.832030 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.831956    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:12.868924 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:55:12.868892    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod286dcd01_9773_4e55_a391_f22a292e5141.slice/crio-b51cc02a2713805bc78748d6d641d16af2370d31435163e6d974c4b90cbec9b1 WatchSource:0}: Error finding container b51cc02a2713805bc78748d6d641d16af2370d31435163e6d974c4b90cbec9b1: Status 404 returned error can't find the container with id b51cc02a2713805bc78748d6d641d16af2370d31435163e6d974c4b90cbec9b1
Oct 30 22:55:12.872509 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:12.872484    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"]
Oct 30 22:55:13.161689 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:13.160874    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"]
Oct 30 22:55:13.169987 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:55:13.169956    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60baf000_5ece_459d_85f8_18e11e72217e.slice/crio-5ebce13b4c45fb81b85e435c01ef4697d3bd1135a5415f59158f829b6c0d1664 WatchSource:0}: Error finding container 5ebce13b4c45fb81b85e435c01ef4697d3bd1135a5415f59158f829b6c0d1664: Status 404 returned error can't find the container with id 5ebce13b4c45fb81b85e435c01ef4697d3bd1135a5415f59158f829b6c0d1664
Oct 30 22:55:13.493720 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:13.492718    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" event={"ID":"60baf000-5ece-459d-85f8-18e11e72217e","Type":"ContainerStarted","Data":"07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d"}
Oct 30 22:55:13.493720 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:13.492790    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" event={"ID":"60baf000-5ece-459d-85f8-18e11e72217e","Type":"ContainerStarted","Data":"5ebce13b4c45fb81b85e435c01ef4697d3bd1135a5415f59158f829b6c0d1664"}
Oct 30 22:55:13.493720 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:13.493597    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:13.501332 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:13.501305    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw" event={"ID":"286dcd01-9773-4e55-a391-f22a292e5141","Type":"ContainerStarted","Data":"55b1650e62f4d5d2dc41a7acfdb9b522154dee2bb57728b6d1137181e0c01a2d"}
Oct 30 22:55:13.501544 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:13.501522    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw" event={"ID":"286dcd01-9773-4e55-a391-f22a292e5141","Type":"ContainerStarted","Data":"b51cc02a2713805bc78748d6d641d16af2370d31435163e6d974c4b90cbec9b1"}
Oct 30 22:55:13.519219 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:13.519173    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" podStartSLOduration=1.5191576709999999 podStartE2EDuration="1.519157671s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:55:13.518493841 +0000 UTC m=+2155.787575356" watchObservedRunningTime="2026-10-30 22:55:13.519157671 +0000 UTC m=+2155.788239194"
Oct 30 22:55:13.552850 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:13.552789    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw" podStartSLOduration=1.552776537 podStartE2EDuration="1.552776537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:55:13.552390433 +0000 UTC m=+2155.821471955" watchObservedRunningTime="2026-10-30 22:55:13.552776537 +0000 UTC m=+2155.821858060"
Oct 30 22:55:13.575903 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:13.575876    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:15.391486 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:55:15.388846    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:55:18.009023 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:18.008989    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:22.576179 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:22.576145    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:22.587778 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:22.587756    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:23.564771 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:23.564746    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:25.479499 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:25.479466    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:40.622935 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:40.622904    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:45.389678 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:55:45.389621    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:55:45.514046 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:45.514019    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:55:47.235644 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:47.235605    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"]
Oct 30 22:55:47.254359 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:47.254334    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"]
Oct 30 22:55:47.264184 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:55:47.264151    5503 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="55b1650e62f4d5d2dc41a7acfdb9b522154dee2bb57728b6d1137181e0c01a2d" cmd=["/bin/sleep","15"]
Oct 30 22:55:47.264308 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:47.264197    5503 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:55:47.264308 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:55:47.264209    5503 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw" podUID="286dcd01-9773-4e55-a391-f22a292e5141" containerName="main" containerID="cri-o://55b1650e62f4d5d2dc41a7acfdb9b522154dee2bb57728b6d1137181e0c01a2d"
Oct 30 22:55:47.264308 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:47.264253    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw" podUID="286dcd01-9773-4e55-a391-f22a292e5141" containerName="main" containerID="cri-o://55b1650e62f4d5d2dc41a7acfdb9b522154dee2bb57728b6d1137181e0c01a2d" gracePeriod=60
Oct 30 22:55:47.316364 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:47.316332    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:47.441623 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:47.441595    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:47.555436 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:47.555340    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw" podUID="286dcd01-9773-4e55-a391-f22a292e5141" containerName="main" probeResult="failure" output="Get \"https://10.133.0.63:8000/health\": dial tcp 10.133.0.63:8000: connect: connection refused"
Oct 30 22:55:47.678182 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:47.678151    5503 generic.go:356] "Generic (PLEG): container finished" podID="286dcd01-9773-4e55-a391-f22a292e5141" containerID="55b1650e62f4d5d2dc41a7acfdb9b522154dee2bb57728b6d1137181e0c01a2d" exitCode=0
Oct 30 22:55:47.678339 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:47.678224    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw" event={"ID":"286dcd01-9773-4e55-a391-f22a292e5141","Type":"ContainerDied","Data":"55b1650e62f4d5d2dc41a7acfdb9b522154dee2bb57728b6d1137181e0c01a2d"}
Oct 30 22:55:48.030091 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.030067    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:48.121354 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.121324    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-tmp-dir\") pod \"286dcd01-9773-4e55-a391-f22a292e5141\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") "
Oct 30 22:55:48.121517 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.121369    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/286dcd01-9773-4e55-a391-f22a292e5141-kube-api-access-2rqzp\" (UniqueName: \"kubernetes.io/projected/286dcd01-9773-4e55-a391-f22a292e5141-kube-api-access-2rqzp\") pod \"286dcd01-9773-4e55-a391-f22a292e5141\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") "
Oct 30 22:55:48.121517 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.121426    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-model-cache\") pod \"286dcd01-9773-4e55-a391-f22a292e5141\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") "
Oct 30 22:55:48.121517 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.121453    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-home\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-home\") pod \"286dcd01-9773-4e55-a391-f22a292e5141\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") "
Oct 30 22:55:48.121661 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.121637    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/286dcd01-9773-4e55-a391-f22a292e5141-tls-certs\" (UniqueName: \"kubernetes.io/secret/286dcd01-9773-4e55-a391-f22a292e5141-tls-certs\") pod \"286dcd01-9773-4e55-a391-f22a292e5141\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") "
Oct 30 22:55:48.121742 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.121691    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-tmp-dir" pod "286dcd01-9773-4e55-a391-f22a292e5141" (UID: "286dcd01-9773-4e55-a391-f22a292e5141"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:48.121742 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.121719    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-model-cache" pod "286dcd01-9773-4e55-a391-f22a292e5141" (UID: "286dcd01-9773-4e55-a391-f22a292e5141"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:48.121742 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.121732    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-home" pod "286dcd01-9773-4e55-a391-f22a292e5141" (UID: "286dcd01-9773-4e55-a391-f22a292e5141"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:48.121900 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.121725    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-dshm\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-dshm\") pod \"286dcd01-9773-4e55-a391-f22a292e5141\" (UID: \"286dcd01-9773-4e55-a391-f22a292e5141\") "
Oct 30 22:55:48.122164 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.122137    5503 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-home\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:48.122451 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.122162    5503 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-tmp-dir\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:48.122451 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.122250    5503 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-model-cache\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:48.123919 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.123894    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-dshm" pod "286dcd01-9773-4e55-a391-f22a292e5141" (UID: "286dcd01-9773-4e55-a391-f22a292e5141"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:48.124056 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.123956    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/286dcd01-9773-4e55-a391-f22a292e5141-tls-certs" pod "286dcd01-9773-4e55-a391-f22a292e5141" (UID: "286dcd01-9773-4e55-a391-f22a292e5141"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:55:48.124056 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.124026    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/286dcd01-9773-4e55-a391-f22a292e5141-kube-api-access-2rqzp" pod "286dcd01-9773-4e55-a391-f22a292e5141" (UID: "286dcd01-9773-4e55-a391-f22a292e5141"). InnerVolumeSpecName "kube-api-access-2rqzp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:48.223557 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.223484    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/286dcd01-9773-4e55-a391-f22a292e5141-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:48.223557 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.223513    5503 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/286dcd01-9773-4e55-a391-f22a292e5141-dshm\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:48.223557 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.223523    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rqzp\" (UniqueName: \"kubernetes.io/projected/286dcd01-9773-4e55-a391-f22a292e5141-kube-api-access-2rqzp\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:48.691140 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.691105    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw" event={"ID":"286dcd01-9773-4e55-a391-f22a292e5141","Type":"ContainerDied","Data":"b51cc02a2713805bc78748d6d641d16af2370d31435163e6d974c4b90cbec9b1"}
Oct 30 22:55:48.691140 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.691132    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"
Oct 30 22:55:48.691772 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.691189    5503 scope.go:122] "RemoveContainer" containerID="55b1650e62f4d5d2dc41a7acfdb9b522154dee2bb57728b6d1137181e0c01a2d"
Oct 30 22:55:48.715525 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.715494    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"]
Oct 30 22:55:48.720705 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:48.720681    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-5f76f74777dprjw"]
Oct 30 22:55:49.855108 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:49.855079    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="286dcd01-9773-4e55-a391-f22a292e5141" path="/var/lib/kubelet/pods/286dcd01-9773-4e55-a391-f22a292e5141/volumes"
Oct 30 22:55:53.116753 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:53.116710    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"]
Oct 30 22:55:53.117261 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:53.117031    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy" containerID="cri-o://f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299" gracePeriod=30
Oct 30 22:55:53.119169 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:53.119146    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"]
Oct 30 22:55:53.119336 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:53.119315    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" containerName="istio-proxy" containerID="cri-o://9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b" gracePeriod=30
Oct 30 22:55:55.292004 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:55.291953    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:55.396764 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:55.396729    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:55:55.514126 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:55.514087    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" podUID="60baf000-5ece-459d-85f8-18e11e72217e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:55:56.710981 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:56.710943    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-rb29g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 22:55:56.711418 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:56.711005    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 22:55:58.363332 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.363310    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:55:58.392155 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.392099    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:55:58.418788 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.418759    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-kube-api-access-5pz49\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-kube-api-access-5pz49\") pod \"aa22b625-db5f-47bf-8975-42293092dd4a\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") "
Oct 30 22:55:58.418963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.418818    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-socket\") pod \"8b80c335-71f6-4e1c-acc5-fb6003572f92\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") "
Oct 30 22:55:58.418963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.418851    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-socket\") pod \"aa22b625-db5f-47bf-8975-42293092dd4a\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") "
Oct 30 22:55:58.420362 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.418910    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-data\") pod \"8b80c335-71f6-4e1c-acc5-fb6003572f92\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") "
Oct 30 22:55:58.420465 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.419354    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-socket" pod "8b80c335-71f6-4e1c-acc5-fb6003572f92" (UID: "8b80c335-71f6-4e1c-acc5-fb6003572f92"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.420465 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.419306    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-socket" pod "aa22b625-db5f-47bf-8975-42293092dd4a" (UID: "aa22b625-db5f-47bf-8975-42293092dd4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.420465 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.419381    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-data" pod "8b80c335-71f6-4e1c-acc5-fb6003572f92" (UID: "8b80c335-71f6-4e1c-acc5-fb6003572f92"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.420465 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420423    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-certs\") pod \"aa22b625-db5f-47bf-8975-42293092dd4a\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") "
Oct 30 22:55:58.420726 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420472    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/aa22b625-db5f-47bf-8975-42293092dd4a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa22b625-db5f-47bf-8975-42293092dd4a-istio-podinfo\") pod \"aa22b625-db5f-47bf-8975-42293092dd4a\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") "
Oct 30 22:55:58.420726 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420545    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-data\") pod \"aa22b625-db5f-47bf-8975-42293092dd4a\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") "
Oct 30 22:55:58.420726 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420579    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-istio-token\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-istio-token\") pod \"aa22b625-db5f-47bf-8975-42293092dd4a\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") "
Oct 30 22:55:58.420887 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420721    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-credential-socket\") pod \"aa22b625-db5f-47bf-8975-42293092dd4a\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") "
Oct 30 22:55:58.420887 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420852    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-envoy\") pod \"8b80c335-71f6-4e1c-acc5-fb6003572f92\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") "
Oct 30 22:55:58.420887 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420859    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-data" pod "aa22b625-db5f-47bf-8975-42293092dd4a" (UID: "aa22b625-db5f-47bf-8975-42293092dd4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.421034 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420891    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-podinfo\") pod \"8b80c335-71f6-4e1c-acc5-fb6003572f92\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") "
Oct 30 22:55:58.421034 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420947    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8b80c335-71f6-4e1c-acc5-fb6003572f92-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b80c335-71f6-4e1c-acc5-fb6003572f92-istiod-ca-cert\") pod \"8b80c335-71f6-4e1c-acc5-fb6003572f92\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") "
Oct 30 22:55:58.421034 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.420983    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-token\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-token\") pod \"8b80c335-71f6-4e1c-acc5-fb6003572f92\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") "
Oct 30 22:55:58.421034 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.421016    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-envoy\") pod \"aa22b625-db5f-47bf-8975-42293092dd4a\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") "
Oct 30 22:55:58.421239 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.421050    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-kube-api-access-xmmmk\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-kube-api-access-xmmmk\") pod \"8b80c335-71f6-4e1c-acc5-fb6003572f92\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") "
Oct 30 22:55:58.421239 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.421083    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-certs\") pod \"8b80c335-71f6-4e1c-acc5-fb6003572f92\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") "
Oct 30 22:55:58.421239 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.421134    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-credential-socket\") pod \"8b80c335-71f6-4e1c-acc5-fb6003572f92\" (UID: \"8b80c335-71f6-4e1c-acc5-fb6003572f92\") "
Oct 30 22:55:58.421239 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.421166    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aa22b625-db5f-47bf-8975-42293092dd4a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa22b625-db5f-47bf-8975-42293092dd4a-istiod-ca-cert\") pod \"aa22b625-db5f-47bf-8975-42293092dd4a\" (UID: \"aa22b625-db5f-47bf-8975-42293092dd4a\") "
Oct 30 22:55:58.421478 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.421267    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-credential-socket" pod "aa22b625-db5f-47bf-8975-42293092dd4a" (UID: "aa22b625-db5f-47bf-8975-42293092dd4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.422339 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.421607    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa22b625-db5f-47bf-8975-42293092dd4a-istiod-ca-cert" pod "aa22b625-db5f-47bf-8975-42293092dd4a" (UID: "aa22b625-db5f-47bf-8975-42293092dd4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:58.422339 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.422102    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b80c335-71f6-4e1c-acc5-fb6003572f92-istiod-ca-cert" pod "8b80c335-71f6-4e1c-acc5-fb6003572f92" (UID: "8b80c335-71f6-4e1c-acc5-fb6003572f92"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:55:58.422339 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.422267    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-kube-api-access-5pz49" pod "aa22b625-db5f-47bf-8975-42293092dd4a" (UID: "aa22b625-db5f-47bf-8975-42293092dd4a"). InnerVolumeSpecName "kube-api-access-5pz49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:58.422787 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.422763    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-certs" pod "aa22b625-db5f-47bf-8975-42293092dd4a" (UID: "aa22b625-db5f-47bf-8975-42293092dd4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.423364 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423340    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-certs" pod "8b80c335-71f6-4e1c-acc5-fb6003572f92" (UID: "8b80c335-71f6-4e1c-acc5-fb6003572f92"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.423547 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423527    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-credential-socket" pod "8b80c335-71f6-4e1c-acc5-fb6003572f92" (UID: "8b80c335-71f6-4e1c-acc5-fb6003572f92"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.423659 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423643    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-istio-token" pod "aa22b625-db5f-47bf-8975-42293092dd4a" (UID: "aa22b625-db5f-47bf-8975-42293092dd4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:58.423767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423694    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pz49\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-kube-api-access-5pz49\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.423767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423713    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.423767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423723    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.423767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423731    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.423767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423739    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.423767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423747    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.423767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423755    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.423767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423762    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b80c335-71f6-4e1c-acc5-fb6003572f92-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.423767 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423770    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.424186 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423778    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.424186 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.423786    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa22b625-db5f-47bf-8975-42293092dd4a-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.424286 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.424162    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa22b625-db5f-47bf-8975-42293092dd4a-istio-podinfo" pod "aa22b625-db5f-47bf-8975-42293092dd4a" (UID: "aa22b625-db5f-47bf-8975-42293092dd4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:55:58.424554 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.424528    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-kube-api-access-xmmmk" pod "8b80c335-71f6-4e1c-acc5-fb6003572f92" (UID: "8b80c335-71f6-4e1c-acc5-fb6003572f92"). InnerVolumeSpecName "kube-api-access-xmmmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:58.425418 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.425397    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-envoy" pod "8b80c335-71f6-4e1c-acc5-fb6003572f92" (UID: "8b80c335-71f6-4e1c-acc5-fb6003572f92"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.425520 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.425413    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-token" pod "8b80c335-71f6-4e1c-acc5-fb6003572f92" (UID: "8b80c335-71f6-4e1c-acc5-fb6003572f92"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:55:58.425852 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.425811    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-podinfo" pod "8b80c335-71f6-4e1c-acc5-fb6003572f92" (UID: "8b80c335-71f6-4e1c-acc5-fb6003572f92"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:55:58.426854 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.426831    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-envoy" pod "aa22b625-db5f-47bf-8975-42293092dd4a" (UID: "aa22b625-db5f-47bf-8975-42293092dd4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:55:58.524427 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.524402    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa22b625-db5f-47bf-8975-42293092dd4a-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.524427 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.524424    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa22b625-db5f-47bf-8975-42293092dd4a-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.524566 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.524432    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.524566 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.524442    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.524566 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.524453    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.524566 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.524461    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa22b625-db5f-47bf-8975-42293092dd4a-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.524566 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.524469    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmmmk\" (UniqueName: \"kubernetes.io/projected/8b80c335-71f6-4e1c-acc5-fb6003572f92-kube-api-access-xmmmk\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:55:58.744725 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.744629    5503 generic.go:356] "Generic (PLEG): container finished" podID="aa22b625-db5f-47bf-8975-42293092dd4a" containerID="9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b" exitCode=0
Oct 30 22:55:58.744725 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.744699    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"
Oct 30 22:55:58.744725 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.744702    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" event={"ID":"aa22b625-db5f-47bf-8975-42293092dd4a","Type":"ContainerDied","Data":"9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b"}
Oct 30 22:55:58.745022 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.744762    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7" event={"ID":"aa22b625-db5f-47bf-8975-42293092dd4a","Type":"ContainerDied","Data":"16411c9d372085a813851a6384deb452754289561d355e717481fd83e67eadc2"}
Oct 30 22:55:58.745022 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.744773    5503 scope.go:122] "RemoveContainer" containerID="9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b"
Oct 30 22:55:58.746735 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.746711    5503 generic.go:356] "Generic (PLEG): container finished" podID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerID="f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299" exitCode=0
Oct 30 22:55:58.746884 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.746764    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"
Oct 30 22:55:58.746884 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.746817    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" event={"ID":"8b80c335-71f6-4e1c-acc5-fb6003572f92","Type":"ContainerDied","Data":"f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299"}
Oct 30 22:55:58.746884 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.746851    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g" event={"ID":"8b80c335-71f6-4e1c-acc5-fb6003572f92","Type":"ContainerDied","Data":"7ca7c42d9a48e9266712a20333a6516dcdb99ff93452908698711c3b11d4ba53"}
Oct 30 22:55:58.755181 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.755106    5503 scope.go:122] "RemoveContainer" containerID="9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b"
Oct 30 22:55:58.755447 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:55:58.755426    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b\": container with ID starting with 9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b not found: ID does not exist" containerID="9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b"
Oct 30 22:55:58.755546 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.755455    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b"} err="failed to get container status \"9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b\": rpc error: code = NotFound desc = could not find container \"9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b\": container with ID starting with 9a7e8aa1120a996a2376f57bf55bc10fd5ab073797db227d22230de8299a049b not found: ID does not exist"
Oct 30 22:55:58.755546 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.755471    5503 scope.go:122] "RemoveContainer" containerID="f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299"
Oct 30 22:55:58.763510 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.763494    5503 scope.go:122] "RemoveContainer" containerID="3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5"
Oct 30 22:55:58.771559 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.771541    5503 scope.go:122] "RemoveContainer" containerID="f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299"
Oct 30 22:55:58.771831 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:55:58.771813    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299\": container with ID starting with f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299 not found: ID does not exist" containerID="f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299"
Oct 30 22:55:58.771831 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.771824    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"]
Oct 30 22:55:58.771927 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.771837    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299"} err="failed to get container status \"f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299\": rpc error: code = NotFound desc = could not find container \"f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299\": container with ID starting with f5e351f31440354df79fbeaa7246c0dc33214ac0122b97bfbe20a8aa6be17299 not found: ID does not exist"
Oct 30 22:55:58.771927 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.771853    5503 scope.go:122] "RemoveContainer" containerID="3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5"
Oct 30 22:55:58.772086 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:55:58.772069    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5\": container with ID starting with 3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5 not found: ID does not exist" containerID="3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5"
Oct 30 22:55:58.772124 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.772091    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5"} err="failed to get container status \"3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5\": rpc error: code = NotFound desc = could not find container \"3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5\": container with ID starting with 3d15b6bce18d03f3db672175068ff576440fe0f8fd1c64dc2c33d7c16b4bc3c5 not found: ID does not exist"
Oct 30 22:55:58.778492 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.778474    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-ks2t7"]
Oct 30 22:55:58.790597 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.790577    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"]
Oct 30 22:55:58.795456 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:58.795438    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-rb29g"]
Oct 30 22:55:59.862179 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:59.862136    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" path="/var/lib/kubelet/pods/8b80c335-71f6-4e1c-acc5-fb6003572f92/volumes"
Oct 30 22:55:59.862758 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:55:59.862740    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" path="/var/lib/kubelet/pods/aa22b625-db5f-47bf-8975-42293092dd4a/volumes"
Oct 30 22:56:02.256906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:02.256870    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" podUID="60baf000-5ece-459d-85f8-18e11e72217e" containerName="main" containerID="cri-o://07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d" gracePeriod=45
Oct 30 22:56:05.514377 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:05.514343    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" podUID="60baf000-5ece-459d-85f8-18e11e72217e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:56:15.389086 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:56:15.389047    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:56:15.514320 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:15.514280    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" podUID="60baf000-5ece-459d-85f8-18e11e72217e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:56:15.514506 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:15.514401    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:56:25.514475 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:25.514438    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" podUID="60baf000-5ece-459d-85f8-18e11e72217e" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:56:25.554679 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:25.554644    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:32.511636 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.511607    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:56:32.525934 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.525870    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60baf000-5ece-459d-85f8-18e11e72217e-kube-api-access-9z7bb\" (UniqueName: \"kubernetes.io/projected/60baf000-5ece-459d-85f8-18e11e72217e-kube-api-access-9z7bb\") pod \"60baf000-5ece-459d-85f8-18e11e72217e\" (UID: \"60baf000-5ece-459d-85f8-18e11e72217e\") "
Oct 30 22:56:32.526072 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.525952    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/60baf000-5ece-459d-85f8-18e11e72217e-tls-certs\" (UniqueName: \"kubernetes.io/secret/60baf000-5ece-459d-85f8-18e11e72217e-tls-certs\") pod \"60baf000-5ece-459d-85f8-18e11e72217e\" (UID: \"60baf000-5ece-459d-85f8-18e11e72217e\") "
Oct 30 22:56:32.528129 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.528097    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60baf000-5ece-459d-85f8-18e11e72217e-tls-certs" pod "60baf000-5ece-459d-85f8-18e11e72217e" (UID: "60baf000-5ece-459d-85f8-18e11e72217e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:56:32.528231 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.528192    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60baf000-5ece-459d-85f8-18e11e72217e-kube-api-access-9z7bb" pod "60baf000-5ece-459d-85f8-18e11e72217e" (UID: "60baf000-5ece-459d-85f8-18e11e72217e"). InnerVolumeSpecName "kube-api-access-9z7bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:56:32.627211 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.627178    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z7bb\" (UniqueName: \"kubernetes.io/projected/60baf000-5ece-459d-85f8-18e11e72217e-kube-api-access-9z7bb\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:32.627211 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.627212    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60baf000-5ece-459d-85f8-18e11e72217e-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:32.925176 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.925146    5503 generic.go:356] "Generic (PLEG): container finished" podID="60baf000-5ece-459d-85f8-18e11e72217e" containerID="07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d" exitCode=0
Oct 30 22:56:32.925379 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.925192    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"
Oct 30 22:56:32.925379 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.925238    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" event={"ID":"60baf000-5ece-459d-85f8-18e11e72217e","Type":"ContainerDied","Data":"07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d"}
Oct 30 22:56:32.925379 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.925298    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx" event={"ID":"60baf000-5ece-459d-85f8-18e11e72217e","Type":"ContainerDied","Data":"5ebce13b4c45fb81b85e435c01ef4697d3bd1135a5415f59158f829b6c0d1664"}
Oct 30 22:56:32.925379 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.925326    5503 scope.go:122] "RemoveContainer" containerID="07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d"
Oct 30 22:56:32.936873 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.935513    5503 scope.go:122] "RemoveContainer" containerID="07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d"
Oct 30 22:56:32.937509 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:56:32.937476    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d\": container with ID starting with 07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d not found: ID does not exist" containerID="07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d"
Oct 30 22:56:32.937607 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.937519    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d"} err="failed to get container status \"07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d\": rpc error: code = NotFound desc = could not find container \"07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d\": container with ID starting with 07a9e109524ae2c39d30900e3eba0d9f00670c3e6146fd4eb6f6e8a86652f19d not found: ID does not exist"
Oct 30 22:56:32.949525 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.949497    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"]
Oct 30 22:56:32.954526 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:32.954502    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc08544b88a8d963ffd553cc1f3ed82d16-kserve-router-schehkgdx"]
Oct 30 22:56:33.854890 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:33.854850    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60baf000-5ece-459d-85f8-18e11e72217e" path="/var/lib/kubelet/pods/60baf000-5ece-459d-85f8-18e11e72217e/volumes"
Oct 30 22:56:37.607645 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:37.607603    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"]
Oct 30 22:56:37.611043 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:37.611017    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:37.718681 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:37.718636    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:40.668273 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:40.668244    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:45.389568 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:56:45.389534    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:56:45.600128 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:45.600099    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:45.703783 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:45.703708    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:45.806226 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:45.806198    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:46.116794 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:46.116757    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:46.216824 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:46.216794    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:47.257491 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:47.257457    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" podUID="30c69508-e130-413a-b829-524a9cafb973" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:56:48.984744 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:48.984720    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:52.403476 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:52.403443    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:52.629456 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:52.629402    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" podUID="30c69508-e130-413a-b829-524a9cafb973" containerName="main" containerID="cri-o://ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8" gracePeriod=45
Oct 30 22:56:54.364511 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:54.364479    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:57.258322 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:57.258241    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" podUID="30c69508-e130-413a-b829-524a9cafb973" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:56:58.382593 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:58.382559    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:56:58.484882 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:56:58.484847    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:07.257951 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:07.257914    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" podUID="30c69508-e130-413a-b829-524a9cafb973" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:57:07.258320 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:07.258012    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:57:15.388048 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:57:15.388005    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:57:17.257543 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:17.257509    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" podUID="30c69508-e130-413a-b829-524a9cafb973" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 22:57:22.880069 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:22.880044    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:57:23.045722 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.045648    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30c69508-e130-413a-b829-524a9cafb973-tls-certs\" (UniqueName: \"kubernetes.io/secret/30c69508-e130-413a-b829-524a9cafb973-tls-certs\") pod \"30c69508-e130-413a-b829-524a9cafb973\" (UID: \"30c69508-e130-413a-b829-524a9cafb973\") "
Oct 30 22:57:23.045722 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.045705    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30c69508-e130-413a-b829-524a9cafb973-kube-api-access-fcd5p\" (UniqueName: \"kubernetes.io/projected/30c69508-e130-413a-b829-524a9cafb973-kube-api-access-fcd5p\") pod \"30c69508-e130-413a-b829-524a9cafb973\" (UID: \"30c69508-e130-413a-b829-524a9cafb973\") "
Oct 30 22:57:23.047888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.047862    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30c69508-e130-413a-b829-524a9cafb973-kube-api-access-fcd5p" pod "30c69508-e130-413a-b829-524a9cafb973" (UID: "30c69508-e130-413a-b829-524a9cafb973"). InnerVolumeSpecName "kube-api-access-fcd5p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:57:23.047888 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.047862    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30c69508-e130-413a-b829-524a9cafb973-tls-certs" pod "30c69508-e130-413a-b829-524a9cafb973" (UID: "30c69508-e130-413a-b829-524a9cafb973"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:57:23.147263 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.147239    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30c69508-e130-413a-b829-524a9cafb973-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:23.147263 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.147262    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcd5p\" (UniqueName: \"kubernetes.io/projected/30c69508-e130-413a-b829-524a9cafb973-kube-api-access-fcd5p\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:57:23.177923 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.177894    5503 generic.go:356] "Generic (PLEG): container finished" podID="30c69508-e130-413a-b829-524a9cafb973" containerID="ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8" exitCode=0
Oct 30 22:57:23.178007 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.177937    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"
Oct 30 22:57:23.178057 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.177938    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" event={"ID":"30c69508-e130-413a-b829-524a9cafb973","Type":"ContainerDied","Data":"ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8"}
Oct 30 22:57:23.178124 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.178094    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m" event={"ID":"30c69508-e130-413a-b829-524a9cafb973","Type":"ContainerDied","Data":"63915edd23b40a240190701b252857831e1bbbbcdacecc8ef1d9c6ff12c587c4"}
Oct 30 22:57:23.178124 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.178118    5503 scope.go:122] "RemoveContainer" containerID="ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8"
Oct 30 22:57:23.186955 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.186938    5503 scope.go:122] "RemoveContainer" containerID="ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8"
Oct 30 22:57:23.187224 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:57:23.187207    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8\": container with ID starting with ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8 not found: ID does not exist" containerID="ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8"
Oct 30 22:57:23.187291 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.187232    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8"} err="failed to get container status \"ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8\": rpc error: code = NotFound desc = could not find container \"ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8\": container with ID starting with ec7eacf2122e393cad863cdcb86b600113cab372734826826ccefbb6e4cfd1d8 not found: ID does not exist"
Oct 30 22:57:23.200651 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.200633    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"]
Oct 30 22:57:23.205071 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.205047    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schezh95m"]
Oct 30 22:57:23.854965 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:23.854934    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30c69508-e130-413a-b829-524a9cafb973" path="/var/lib/kubelet/pods/30c69508-e130-413a-b829-524a9cafb973/volumes"
Oct 30 22:57:27.311811 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:27.311780    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:27.770997 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:27.770964    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:27.874255 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:27.874221    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:27.976906 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:27.976875    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:30.630639 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:30.630604    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:34.339806 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:34.339763    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:35.583846 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:35.583814    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:36.365681 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.365633    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:36.583629 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.583594    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"]
Oct 30 22:57:36.584220 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584193    5503 cpu_manager.go:414] "containerMap: removing container" podUID="30c69508-e130-413a-b829-524a9cafb973" containerName="main"
Oct 30 22:57:36.584220 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584217    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30c69508-e130-413a-b829-524a9cafb973" containerName="main"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584239    5503 cpu_manager.go:414] "containerMap: removing container" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584244    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584254    5503 cpu_manager.go:414] "containerMap: removing container" podUID="286dcd01-9773-4e55-a391-f22a292e5141" containerName="main"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584260    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="286dcd01-9773-4e55-a391-f22a292e5141" containerName="main"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584267    5503 cpu_manager.go:414] "containerMap: removing container" podUID="60baf000-5ece-459d-85f8-18e11e72217e" containerName="main"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584271    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60baf000-5ece-459d-85f8-18e11e72217e" containerName="main"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584284    5503 cpu_manager.go:414] "containerMap: removing container" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" containerName="istio-proxy"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584314    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" containerName="istio-proxy"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584331    5503 cpu_manager.go:414] "containerMap: removing container" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584336    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584406    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584415    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="30c69508-e130-413a-b829-524a9cafb973" containerName="main"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584423    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="286dcd01-9773-4e55-a391-f22a292e5141" containerName="main"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584427    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="60baf000-5ece-459d-85f8-18e11e72217e" containerName="main"
Oct 30 22:57:36.584740 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.584434    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="aa22b625-db5f-47bf-8975-42293092dd4a" containerName="istio-proxy"
Oct 30 22:57:36.589196 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.589163    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:57:36.596301 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.596276    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"kube-root-ca.crt\""
Oct 30 22:57:36.596411 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.596330    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"openshift-service-ca.crt\""
Oct 30 22:57:36.596457 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.596435    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa-dockercfg-ks9qg\""
Oct 30 22:57:36.596603 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.596581    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs\""
Oct 30 22:57:36.596914 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.596888    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"]
Oct 30 22:57:36.596999 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.596935    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-5bc7dccd\"/\"default-dockercfg-qhs6z\""
Oct 30 22:57:36.630403 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.630317    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:36.660588 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.660558    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74b31522-5e37-4daa-a848-a95404f098b9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp\" (UID: \"74b31522-5e37-4daa-a848-a95404f098b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:57:36.660775 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.660617    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w7rb\" (UniqueName: \"kubernetes.io/projected/74b31522-5e37-4daa-a848-a95404f098b9-kube-api-access-6w7rb\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp\" (UID: \"74b31522-5e37-4daa-a848-a95404f098b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:57:36.761493 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.761402    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w7rb\" (UniqueName: \"kubernetes.io/projected/74b31522-5e37-4daa-a848-a95404f098b9-kube-api-access-6w7rb\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp\" (UID: \"74b31522-5e37-4daa-a848-a95404f098b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:57:36.761739 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.761612    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74b31522-5e37-4daa-a848-a95404f098b9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp\" (UID: \"74b31522-5e37-4daa-a848-a95404f098b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:57:36.764385 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.764361    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74b31522-5e37-4daa-a848-a95404f098b9-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp\" (UID: \"74b31522-5e37-4daa-a848-a95404f098b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:57:36.774107 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.774083    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w7rb\" (UniqueName: \"kubernetes.io/projected/74b31522-5e37-4daa-a848-a95404f098b9-kube-api-access-6w7rb\") pod \"llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp\" (UID: \"74b31522-5e37-4daa-a848-a95404f098b9\") " pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:57:36.904991 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:36.904912    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:57:37.185280 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:37.183552    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"]
Oct 30 22:57:37.190784 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:57:37.190753    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b31522_5e37_4daa_a848_a95404f098b9.slice/crio-b637a723342de7551f5d34d2c87fda9d05fc6fe9b7209a5c662287f3ff058b58 WatchSource:0}: Error finding container b637a723342de7551f5d34d2c87fda9d05fc6fe9b7209a5c662287f3ff058b58: Status 404 returned error can't find the container with id b637a723342de7551f5d34d2c87fda9d05fc6fe9b7209a5c662287f3ff058b58
Oct 30 22:57:37.256259 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:37.256224    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" event={"ID":"74b31522-5e37-4daa-a848-a95404f098b9","Type":"ContainerStarted","Data":"b637a723342de7551f5d34d2c87fda9d05fc6fe9b7209a5c662287f3ff058b58"}
Oct 30 22:57:38.263579 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:38.263547    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" event={"ID":"74b31522-5e37-4daa-a848-a95404f098b9","Type":"ContainerStarted","Data":"7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e"}
Oct 30 22:57:38.263992 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:38.263618    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:57:38.283525 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:38.283477    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" podStartSLOduration=2.283465684 podStartE2EDuration="2.283465684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:57:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:57:38.282192293 +0000 UTC m=+2300.551273818" watchObservedRunningTime="2026-10-30 22:57:38.283465684 +0000 UTC m=+2300.552547207"
Oct 30 22:57:38.516743 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:38.512473    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:40.480950 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:40.480918    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:40.585453 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:40.585423    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:40.804078 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:57:40.803986    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:57:45.388870 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:57:45.388835    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:58:06.517075 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:06.517005    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:08.310618 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.310567    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"]
Oct 30 22:58:08.311123 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.311106    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="8b80c335-71f6-4e1c-acc5-fb6003572f92" containerName="istio-proxy"
Oct 30 22:58:08.314843 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.314814    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.327992 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.327960    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"]
Oct 30 22:58:08.331210 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.331191    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"]
Oct 30 22:58:08.334508 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.334482    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.348020 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.347996    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"]
Oct 30 22:58:08.446548 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446520    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.446548 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446550    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.446753 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446568    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.446753 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446590    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.446753 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446714    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd057bcc-31cc-4a62-9093-47fe2363ac11-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.446848 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446763    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.446848 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446786    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55bda32-45f0-405e-be27-66b3eeefd980-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.446848 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446809    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.446971 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446869    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gskp5\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-kube-api-access-gskp5\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.446971 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446900    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.446971 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446922    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55bda32-45f0-405e-be27-66b3eeefd980-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.446971 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446954    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zdnb\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-kube-api-access-4zdnb\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.447170 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.446980    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.447170 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.447004    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.447170 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.447026    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.447170 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.447058    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.447170 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.447106    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.447170 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.447138    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.547504 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547477    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.547504 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547504    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.547640 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547522    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.547640 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547540    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.547781 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547700    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd057bcc-31cc-4a62-9093-47fe2363ac11-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.547781 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547751    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.547887 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547831    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55bda32-45f0-405e-be27-66b3eeefd980-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.547887 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547862    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.547887 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547879    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.548055 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547915    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.548055 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547927    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.548055 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547917    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gskp5\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-kube-api-access-gskp5\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.548055 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.547990    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.548055 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.548020    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55bda32-45f0-405e-be27-66b3eeefd980-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.548313 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.548073    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdnb\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-kube-api-access-4zdnb\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.548313 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.548097    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.548313 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.548137    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.548313 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.548170    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.548313 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.548203    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.548313 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.548239    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.548809 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.548302    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.548809 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.548554    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd057bcc-31cc-4a62-9093-47fe2363ac11-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.549029 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.549005    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55bda32-45f0-405e-be27-66b3eeefd980-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.549097 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.549008    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.549435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.549249    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.549435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.549326    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.549435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.549370    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.549435 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.549398    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.550250 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.550220    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.550451 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.550426    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.550588 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.550569    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.550706 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.550661    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55bda32-45f0-405e-be27-66b3eeefd980-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.561840 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.561781    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.561840 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.561803    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.562126 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.562110    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zdnb\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-kube-api-access-4zdnb\") pod \"openshift-ai-inference-openshift-default-86b669b864-fdlwm\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.562211 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.562194    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gskp5\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-kube-api-access-gskp5\") pod \"openshift-ai-inference-openshift-default-86b669b864-sf89h\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.629623 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.629607    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:08.650097 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.650074    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:08.914073 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.914051    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"]
Oct 30 22:58:08.915314 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:58:08.915288    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode55bda32_45f0_405e_be27_66b3eeefd980.slice/crio-f895a673437d385fcc1596789c52514d73dfeb3e88298e52e8062689dd71e190 WatchSource:0}: Error finding container f895a673437d385fcc1596789c52514d73dfeb3e88298e52e8062689dd71e190: Status 404 returned error can't find the container with id f895a673437d385fcc1596789c52514d73dfeb3e88298e52e8062689dd71e190
Oct 30 22:58:08.917532 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.917502    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:58:08.917612 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.917565    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:58:08.917612 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.917594    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:58:08.936543 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.936524    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"]
Oct 30 22:58:08.940584 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:58:08.940562    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd057bcc_31cc_4a62_9093_47fe2363ac11.slice/crio-171a36afae649262061fe42a84ccd8a8f9546a6f954f1520c569985aa216f0ec WatchSource:0}: Error finding container 171a36afae649262061fe42a84ccd8a8f9546a6f954f1520c569985aa216f0ec: Status 404 returned error can't find the container with id 171a36afae649262061fe42a84ccd8a8f9546a6f954f1520c569985aa216f0ec
Oct 30 22:58:08.942563 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.942538    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:58:08.942630 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.942594    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:58:08.942630 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:08.942624    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:58:09.270059 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.269976    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 22:58:09.457191 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.457147    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" event={"ID":"bd057bcc-31cc-4a62-9093-47fe2363ac11","Type":"ContainerStarted","Data":"4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4"}
Oct 30 22:58:09.457625 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.457208    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" event={"ID":"bd057bcc-31cc-4a62-9093-47fe2363ac11","Type":"ContainerStarted","Data":"171a36afae649262061fe42a84ccd8a8f9546a6f954f1520c569985aa216f0ec"}
Oct 30 22:58:09.458948 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.458922    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" event={"ID":"e55bda32-45f0-405e-be27-66b3eeefd980","Type":"ContainerStarted","Data":"5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6"}
Oct 30 22:58:09.459065 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.458970    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" event={"ID":"e55bda32-45f0-405e-be27-66b3eeefd980","Type":"ContainerStarted","Data":"f895a673437d385fcc1596789c52514d73dfeb3e88298e52e8062689dd71e190"}
Oct 30 22:58:09.481842 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.481773    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" podStartSLOduration=1.481761484 podStartE2EDuration="1.481761484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:09.480049217 +0000 UTC m=+2331.749130739" watchObservedRunningTime="2026-10-30 22:58:09.481761484 +0000 UTC m=+2331.750843007"
Oct 30 22:58:09.631351 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.631319    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:09.631790 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.631763    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-sf89h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:58:09.631961 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.631936    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:58:09.650758 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.650725    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:09.652050 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.652021    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fdlwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:58:09.652222 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:09.652204    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:58:10.641690 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:10.641052    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-sf89h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:58:10.641690 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:10.641113    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:58:10.650911 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:10.650884    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fdlwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:58:10.651033 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:10.650927    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:58:11.587560 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:11.587515    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:11.630692 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:11.630060    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-sf89h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:58:11.630692 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:11.630123    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:58:11.651213 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:11.651178    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fdlwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:58:11.651557 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:11.651222    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:58:11.701569 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:11.701539    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:12.631064 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:12.631025    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-sf89h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:58:12.631338 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:12.631310    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:58:12.651963 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:12.651933    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fdlwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:58:12.652330 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:12.651982    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:58:13.634054 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:13.634019    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-sf89h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:58:13.634983 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:13.634953    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:58:13.652047 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:13.651964    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fdlwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:58:13.652047 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:13.652002    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:58:14.630521 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:14.630479    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-sf89h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Oct 30 22:58:14.630828 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:14.630788    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Oct 30 22:58:14.650832 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:14.650793    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fdlwm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused" start-of-body=
Oct 30 22:58:14.651017 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:14.650882    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.67:15021/healthz/ready\": dial tcp 10.133.0.67:15021: connect: connection refused"
Oct 30 22:58:15.389057 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:58:15.389017    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:58:15.633987 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:15.633964    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:15.634349 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:15.634326    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:15.635116 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:15.635101    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 22:58:15.654382 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:15.654325    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:15.654607 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:15.654593    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:15.655291 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:15.655278    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 22:58:15.661256 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:15.661221    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" podStartSLOduration=7.661209776 podStartE2EDuration="7.661209776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:09.522275312 +0000 UTC m=+2331.791356835" watchObservedRunningTime="2026-10-30 22:58:15.661209776 +0000 UTC m=+2337.930291298"
Oct 30 22:58:16.639490 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:16.639459    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:18.469448 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.469418    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:18.642244 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.642202    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"]
Oct 30 22:58:18.647459 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.647424    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:18.655942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.655920    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisvc-model-fb-opt-125m-route-87882a8e-epp-sa-dockercfg-jb45v\""
Oct 30 22:58:18.656277 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.656261    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\""
Oct 30 22:58:18.657095 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.657002    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"]
Oct 30 22:58:18.657384 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.657365    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-j7xwg\""
Oct 30 22:58:18.657583 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.657561    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\""
Oct 30 22:58:18.657773 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.657754    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Oct 30 22:58:18.698407 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.698368    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:18.781972 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.781944    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e9a2fd4-5101-4811-a470-1312458e4b41-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8\" (UID: \"1e9a2fd4-5101-4811-a470-1312458e4b41\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:18.782092 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.782012    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgzsl\" (UniqueName: \"kubernetes.io/projected/1e9a2fd4-5101-4811-a470-1312458e4b41-kube-api-access-dgzsl\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8\" (UID: \"1e9a2fd4-5101-4811-a470-1312458e4b41\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:18.882942 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.882915    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgzsl\" (UniqueName: \"kubernetes.io/projected/1e9a2fd4-5101-4811-a470-1312458e4b41-kube-api-access-dgzsl\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8\" (UID: \"1e9a2fd4-5101-4811-a470-1312458e4b41\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:18.883049 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.882983    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e9a2fd4-5101-4811-a470-1312458e4b41-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8\" (UID: \"1e9a2fd4-5101-4811-a470-1312458e4b41\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:18.885527 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.885508    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e9a2fd4-5101-4811-a470-1312458e4b41-tls-certs\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8\" (UID: \"1e9a2fd4-5101-4811-a470-1312458e4b41\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:18.895707 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.895632    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgzsl\" (UniqueName: \"kubernetes.io/projected/1e9a2fd4-5101-4811-a470-1312458e4b41-kube-api-access-dgzsl\") pod \"llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8\" (UID: \"1e9a2fd4-5101-4811-a470-1312458e4b41\") " pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:18.968163 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:18.968127    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:19.255109 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:19.255083    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"]
Oct 30 22:58:19.256823 ip-10-0-128-97 kubenswrapper[5503]: W1030 22:58:19.256790    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e9a2fd4_5101_4811_a470_1312458e4b41.slice/crio-0f96d9b9ea5009b10f874880c72e9cc08b8e7be09aebc32222901c465f01578b WatchSource:0}: Error finding container 0f96d9b9ea5009b10f874880c72e9cc08b8e7be09aebc32222901c465f01578b: Status 404 returned error can't find the container with id 0f96d9b9ea5009b10f874880c72e9cc08b8e7be09aebc32222901c465f01578b
Oct 30 22:58:19.529762 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:19.524202    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" event={"ID":"1e9a2fd4-5101-4811-a470-1312458e4b41","Type":"ContainerStarted","Data":"b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a"}
Oct 30 22:58:19.529762 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:19.524285    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" event={"ID":"1e9a2fd4-5101-4811-a470-1312458e4b41","Type":"ContainerStarted","Data":"0f96d9b9ea5009b10f874880c72e9cc08b8e7be09aebc32222901c465f01578b"}
Oct 30 22:58:19.529762 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:19.524746    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:19.556730 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:19.553352    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" podStartSLOduration=1.553333285 podStartE2EDuration="1.553333285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:19.549531872 +0000 UTC m=+2341.818613396" watchObservedRunningTime="2026-10-30 22:58:19.553333285 +0000 UTC m=+2341.822414808"
Oct 30 22:58:21.105388 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:21.105356    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:30.688061 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:30.687972    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:35.725246 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:35.725217    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:37.966409 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:37.966379    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:39.090105 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:39.090066    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:40.865452 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:40.865421    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:44.887544 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:44.887509    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:45.388658 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:58:45.388631    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:58:45.644068 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:45.643995    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:58:50.530766 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:50.530739    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 22:58:54.212450 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:58:54.212401    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:00.079748 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:59:00.079715    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:00.184938 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:59:00.184891    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:15.389244 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:59:15.389205    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:59:22.024440 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:59:22.024401    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:22.127829 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:59:22.127769    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:30.740893 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:59:30.740837    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:35.675506 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:59:35.675450    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 22:59:45.389353 ip-10-0-128-97 kubenswrapper[5503]: E1030 22:59:45.389315    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 22:59:50.924561 ip-10-0-128-97 kubenswrapper[5503]: I1030 22:59:50.924509    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:13.202476 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:13.202434    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"]
Oct 30 23:00:13.210872 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:13.210840    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:13.321493 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:13.320897    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:15.388885 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:00:15.388852    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:00:20.530527 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:20.530494    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" podUID="1e9a2fd4-5101-4811-a470-1312458e4b41" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:00:28.224096 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:28.224050    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" podUID="1e9a2fd4-5101-4811-a470-1312458e4b41" containerName="main" containerID="cri-o://b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a" gracePeriod=45
Oct 30 23:00:30.531182 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:30.531142    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" podUID="1e9a2fd4-5101-4811-a470-1312458e4b41" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:00:30.803092 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:30.803006    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:40.530737 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:40.530702    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" podUID="1e9a2fd4-5101-4811-a470-1312458e4b41" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:00:40.531176 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:40.530791    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 23:00:42.491708 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:42.491657    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:42.592116 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:42.592086    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:45.388596 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:00:45.388557    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:00:50.530989 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:50.530955    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" podUID="1e9a2fd4-5101-4811-a470-1312458e4b41" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:00:50.978187 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:50.978159    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:00:58.463761 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:58.463736    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 23:00:58.602647 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:58.602610    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e9a2fd4-5101-4811-a470-1312458e4b41-kube-api-access-dgzsl\" (UniqueName: \"kubernetes.io/projected/1e9a2fd4-5101-4811-a470-1312458e4b41-kube-api-access-dgzsl\") pod \"1e9a2fd4-5101-4811-a470-1312458e4b41\" (UID: \"1e9a2fd4-5101-4811-a470-1312458e4b41\") "
Oct 30 23:00:58.602864 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:58.602791    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e9a2fd4-5101-4811-a470-1312458e4b41-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e9a2fd4-5101-4811-a470-1312458e4b41-tls-certs\") pod \"1e9a2fd4-5101-4811-a470-1312458e4b41\" (UID: \"1e9a2fd4-5101-4811-a470-1312458e4b41\") "
Oct 30 23:00:58.604997 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:58.604971    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e9a2fd4-5101-4811-a470-1312458e4b41-tls-certs" pod "1e9a2fd4-5101-4811-a470-1312458e4b41" (UID: "1e9a2fd4-5101-4811-a470-1312458e4b41"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:00:58.605117 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:58.605005    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e9a2fd4-5101-4811-a470-1312458e4b41-kube-api-access-dgzsl" pod "1e9a2fd4-5101-4811-a470-1312458e4b41" (UID: "1e9a2fd4-5101-4811-a470-1312458e4b41"). InnerVolumeSpecName "kube-api-access-dgzsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:00:58.703713 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:58.703688    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgzsl\" (UniqueName: \"kubernetes.io/projected/1e9a2fd4-5101-4811-a470-1312458e4b41-kube-api-access-dgzsl\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:58.703713 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:58.703711    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e9a2fd4-5101-4811-a470-1312458e4b41-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:59.298501 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.298467    5503 generic.go:356] "Generic (PLEG): container finished" podID="1e9a2fd4-5101-4811-a470-1312458e4b41" containerID="b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a" exitCode=0
Oct 30 23:00:59.298758 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.298518    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" event={"ID":"1e9a2fd4-5101-4811-a470-1312458e4b41","Type":"ContainerDied","Data":"b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a"}
Oct 30 23:00:59.298758 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.298528    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"
Oct 30 23:00:59.298758 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.298589    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8" event={"ID":"1e9a2fd4-5101-4811-a470-1312458e4b41","Type":"ContainerDied","Data":"0f96d9b9ea5009b10f874880c72e9cc08b8e7be09aebc32222901c465f01578b"}
Oct 30 23:00:59.298758 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.298611    5503 scope.go:122] "RemoveContainer" containerID="b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a"
Oct 30 23:00:59.308360 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.308342    5503 scope.go:122] "RemoveContainer" containerID="b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a"
Oct 30 23:00:59.308628 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:00:59.308609    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a\": container with ID starting with b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a not found: ID does not exist" containerID="b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a"
Oct 30 23:00:59.308698 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.308635    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a"} err="failed to get container status \"b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a\": rpc error: code = NotFound desc = could not find container \"b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a\": container with ID starting with b6326bd046635e8fae0952a4593bced92b2c1df4b1dad21a8caea46ed888641a not found: ID does not exist"
Oct 30 23:00:59.321057 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.321032    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"]
Oct 30 23:00:59.325928 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.325906    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvca690bbc929faec8bc98c767f16c003c1-kserve-router-sche6drs8"]
Oct 30 23:00:59.854621 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:00:59.854598    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e9a2fd4-5101-4811-a470-1312458e4b41" path="/var/lib/kubelet/pods/1e9a2fd4-5101-4811-a470-1312458e4b41/volumes"
Oct 30 23:01:03.484834 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:03.484805    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:03.591928 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:03.591903    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:09.726701 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:09.726651    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:15.389368 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:01:15.389331    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:01:15.429658 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:15.429635    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:15.542489 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:15.542466    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:35.806389 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:35.806310    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:35.816403 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:35.816367    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"]
Oct 30 23:01:35.909952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:35.909920    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:39.269564 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:39.269533    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" podUID="74b31522-5e37-4daa-a848-a95404f098b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:01:40.845295 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:40.845241    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:45.389864 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:01:45.389824    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:01:49.269660 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:49.269627    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" podUID="74b31522-5e37-4daa-a848-a95404f098b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:01:50.836376 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:50.836337    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" podUID="74b31522-5e37-4daa-a848-a95404f098b9" containerName="main" containerID="cri-o://7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e" gracePeriod=45
Oct 30 23:01:51.045261 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:51.045235    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:01:59.270441 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:59.270399    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" podUID="74b31522-5e37-4daa-a848-a95404f098b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:01:59.270843 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:01:59.270517    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 23:02:00.769380 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:00.769348    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:09.269477 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:09.269434    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" podUID="74b31522-5e37-4daa-a848-a95404f098b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:02:15.388560 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:02:15.388524    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:02:19.269776 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:19.269744    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" podUID="74b31522-5e37-4daa-a848-a95404f098b9" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:02:21.079512 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.079487    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 23:02:21.229321 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.229244    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/74b31522-5e37-4daa-a848-a95404f098b9-tls-certs\" (UniqueName: \"kubernetes.io/secret/74b31522-5e37-4daa-a848-a95404f098b9-tls-certs\") pod \"74b31522-5e37-4daa-a848-a95404f098b9\" (UID: \"74b31522-5e37-4daa-a848-a95404f098b9\") "
Oct 30 23:02:21.229321 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.229298    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74b31522-5e37-4daa-a848-a95404f098b9-kube-api-access-6w7rb\" (UniqueName: \"kubernetes.io/projected/74b31522-5e37-4daa-a848-a95404f098b9-kube-api-access-6w7rb\") pod \"74b31522-5e37-4daa-a848-a95404f098b9\" (UID: \"74b31522-5e37-4daa-a848-a95404f098b9\") "
Oct 30 23:02:21.231485 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.231456    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74b31522-5e37-4daa-a848-a95404f098b9-kube-api-access-6w7rb" pod "74b31522-5e37-4daa-a848-a95404f098b9" (UID: "74b31522-5e37-4daa-a848-a95404f098b9"). InnerVolumeSpecName "kube-api-access-6w7rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:02:21.231605 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.231456    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74b31522-5e37-4daa-a848-a95404f098b9-tls-certs" pod "74b31522-5e37-4daa-a848-a95404f098b9" (UID: "74b31522-5e37-4daa-a848-a95404f098b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:02:21.330333 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.330298    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74b31522-5e37-4daa-a848-a95404f098b9-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:02:21.330333 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.330330    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w7rb\" (UniqueName: \"kubernetes.io/projected/74b31522-5e37-4daa-a848-a95404f098b9-kube-api-access-6w7rb\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:02:21.692861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.692830    5503 generic.go:356] "Generic (PLEG): container finished" podID="74b31522-5e37-4daa-a848-a95404f098b9" containerID="7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e" exitCode=0
Oct 30 23:02:21.693000 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.692891    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"
Oct 30 23:02:21.693000 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.692911    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" event={"ID":"74b31522-5e37-4daa-a848-a95404f098b9","Type":"ContainerDied","Data":"7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e"}
Oct 30 23:02:21.693000 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.692989    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp" event={"ID":"74b31522-5e37-4daa-a848-a95404f098b9","Type":"ContainerDied","Data":"b637a723342de7551f5d34d2c87fda9d05fc6fe9b7209a5c662287f3ff058b58"}
Oct 30 23:02:21.693184 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.693011    5503 scope.go:122] "RemoveContainer" containerID="7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e"
Oct 30 23:02:21.702653 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.702635    5503 scope.go:122] "RemoveContainer" containerID="7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e"
Oct 30 23:02:21.702938 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:02:21.702919    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e\": container with ID starting with 7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e not found: ID does not exist" containerID="7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e"
Oct 30 23:02:21.702984 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.702947    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e"} err="failed to get container status \"7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e\": rpc error: code = NotFound desc = could not find container \"7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e\": container with ID starting with 7d958b9da3c9f579588cace5a371c8817f3692fc0e9e7095d929ae3855c4061e not found: ID does not exist"
Oct 30 23:02:21.716756 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.716729    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"]
Oct 30 23:02:21.721694 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.721660    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-5bc7dccd/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-sche56jhp"]
Oct 30 23:02:21.855206 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:21.855181    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74b31522-5e37-4daa-a848-a95404f098b9" path="/var/lib/kubelet/pods/74b31522-5e37-4daa-a848-a95404f098b9/volumes"
Oct 30 23:02:26.098535 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:26.098504    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:26.202038 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:26.202010    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:30.863053 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:30.863023    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:31.076527 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.076483    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"]
Oct 30 23:02:31.077155 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.077133    5503 cpu_manager.go:414] "containerMap: removing container" podUID="1e9a2fd4-5101-4811-a470-1312458e4b41" containerName="main"
Oct 30 23:02:31.077255 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.077158    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e9a2fd4-5101-4811-a470-1312458e4b41" containerName="main"
Oct 30 23:02:31.077255 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.077189    5503 cpu_manager.go:414] "containerMap: removing container" podUID="74b31522-5e37-4daa-a848-a95404f098b9" containerName="main"
Oct 30 23:02:31.077255 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.077197    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74b31522-5e37-4daa-a848-a95404f098b9" containerName="main"
Oct 30 23:02:31.077413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.077294    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e9a2fd4-5101-4811-a470-1312458e4b41" containerName="main"
Oct 30 23:02:31.077413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.077310    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="74b31522-5e37-4daa-a848-a95404f098b9" containerName="main"
Oct 30 23:02:31.082287 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.082261    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:02:31.088456 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.088432    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa-dockercfg-2dsmh\""
Oct 30 23:02:31.088605 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.088588    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"openshift-service-ca.crt\""
Oct 30 23:02:31.089384 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.089360    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"default-dockercfg-bdrmn\""
Oct 30 23:02:31.089499 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.089485    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs\""
Oct 30 23:02:31.089570 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.089554    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-733055c7\"/\"kube-root-ca.crt\""
Oct 30 23:02:31.091624 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.091430    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"]
Oct 30 23:02:31.103910 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.103878    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh6q5\" (UniqueName: \"kubernetes.io/projected/4c6f2df0-c745-434f-b256-f6c971020ec3-kube-api-access-kh6q5\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6\" (UID: \"4c6f2df0-c745-434f-b256-f6c971020ec3\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:02:31.104081 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.104059    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c6f2df0-c745-434f-b256-f6c971020ec3-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6\" (UID: \"4c6f2df0-c745-434f-b256-f6c971020ec3\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:02:31.135435 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.135372    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:31.205287 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.205261    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh6q5\" (UniqueName: \"kubernetes.io/projected/4c6f2df0-c745-434f-b256-f6c971020ec3-kube-api-access-kh6q5\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6\" (UID: \"4c6f2df0-c745-434f-b256-f6c971020ec3\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:02:31.205401 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.205381    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c6f2df0-c745-434f-b256-f6c971020ec3-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6\" (UID: \"4c6f2df0-c745-434f-b256-f6c971020ec3\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:02:31.208203 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.208182    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c6f2df0-c745-434f-b256-f6c971020ec3-tls-certs\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6\" (UID: \"4c6f2df0-c745-434f-b256-f6c971020ec3\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:02:31.221613 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.221589    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh6q5\" (UniqueName: \"kubernetes.io/projected/4c6f2df0-c745-434f-b256-f6c971020ec3-kube-api-access-kh6q5\") pod \"llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6\" (UID: \"4c6f2df0-c745-434f-b256-f6c971020ec3\") " pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:02:31.398941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.398869    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:02:31.678909 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.678887    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"]
Oct 30 23:02:31.681614 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:02:31.681588    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c6f2df0_c745_434f_b256_f6c971020ec3.slice/crio-0fa98c6e1b15707768ed1cc7a32ad176abfdf47fbec69c3a2e749b974e34e82d WatchSource:0}: Error finding container 0fa98c6e1b15707768ed1cc7a32ad176abfdf47fbec69c3a2e749b974e34e82d: Status 404 returned error can't find the container with id 0fa98c6e1b15707768ed1cc7a32ad176abfdf47fbec69c3a2e749b974e34e82d
Oct 30 23:02:31.746464 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:31.746437    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" event={"ID":"4c6f2df0-c745-434f-b256-f6c971020ec3","Type":"ContainerStarted","Data":"0fa98c6e1b15707768ed1cc7a32ad176abfdf47fbec69c3a2e749b974e34e82d"}
Oct 30 23:02:32.763693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:32.763341    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" event={"ID":"4c6f2df0-c745-434f-b256-f6c971020ec3","Type":"ContainerStarted","Data":"c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d"}
Oct 30 23:02:32.767687 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:32.764252    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:02:32.793691 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:32.793238    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" podStartSLOduration=1.793206247 podStartE2EDuration="1.793206247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:02:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:02:32.793170482 +0000 UTC m=+2595.062252003" watchObservedRunningTime="2026-10-30 23:02:32.793206247 +0000 UTC m=+2595.062287770"
Oct 30 23:02:33.647693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:33.647453    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:37.107336 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:37.107292    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:39.905093 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:39.905063    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:40.025354 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:40.025326    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:45.389197 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:02:45.389153    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:02:50.915111 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:50.915082    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:02:51.138589 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:02:51.138547    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:00.814682 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:00.814624    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:04.795551 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:04.795525    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:03:15.389440 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:03:15.389391    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:03:18.409788 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:18.409752    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:18.660001 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:18.659976    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:20.696467 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:20.696427    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:45.389356 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:03:45.389320    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:03:50.970770 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:50.970720    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:51.146864 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:51.146832    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:52.627689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:52.626454    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"]
Oct 30 23:03:52.724692 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:52.724206    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:52.837695 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:52.837268    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:03:54.796182 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:03:54.796143    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:04:00.878617 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:00.878582    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:04:04.796079 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:04.796040    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:04:07.837678 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:07.837601    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerName="main" containerID="cri-o://c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d" gracePeriod=45
Oct 30 23:04:14.796192 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:14.796148    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:04:14.796622 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:14.796282    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:04:15.388452 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:04:15.388422    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:04:24.796545 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:24.796510    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:04:34.796336 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:34.796261    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:04:38.081286 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.081258    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:04:38.161468 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.161445    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c6f2df0-c745-434f-b256-f6c971020ec3-kube-api-access-kh6q5\" (UniqueName: \"kubernetes.io/projected/4c6f2df0-c745-434f-b256-f6c971020ec3-kube-api-access-kh6q5\") pod \"4c6f2df0-c745-434f-b256-f6c971020ec3\" (UID: \"4c6f2df0-c745-434f-b256-f6c971020ec3\") "
Oct 30 23:04:38.161579 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.161540    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c6f2df0-c745-434f-b256-f6c971020ec3-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c6f2df0-c745-434f-b256-f6c971020ec3-tls-certs\") pod \"4c6f2df0-c745-434f-b256-f6c971020ec3\" (UID: \"4c6f2df0-c745-434f-b256-f6c971020ec3\") "
Oct 30 23:04:38.163764 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.163743    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c6f2df0-c745-434f-b256-f6c971020ec3-kube-api-access-kh6q5" pod "4c6f2df0-c745-434f-b256-f6c971020ec3" (UID: "4c6f2df0-c745-434f-b256-f6c971020ec3"). InnerVolumeSpecName "kube-api-access-kh6q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:04:38.163857 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.163780    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c6f2df0-c745-434f-b256-f6c971020ec3-tls-certs" pod "4c6f2df0-c745-434f-b256-f6c971020ec3" (UID: "4c6f2df0-c745-434f-b256-f6c971020ec3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:04:38.262944 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.262914    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh6q5\" (UniqueName: \"kubernetes.io/projected/4c6f2df0-c745-434f-b256-f6c971020ec3-kube-api-access-kh6q5\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:04:38.262944 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.262940    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c6f2df0-c745-434f-b256-f6c971020ec3-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:04:38.434836 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.434784    5503 generic.go:356] "Generic (PLEG): container finished" podID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerID="c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d" exitCode=0
Oct 30 23:04:38.434836 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.434827    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"
Oct 30 23:04:38.434961 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.434865    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" event={"ID":"4c6f2df0-c745-434f-b256-f6c971020ec3","Type":"ContainerDied","Data":"c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d"}
Oct 30 23:04:38.434961 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.434915    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6" event={"ID":"4c6f2df0-c745-434f-b256-f6c971020ec3","Type":"ContainerDied","Data":"0fa98c6e1b15707768ed1cc7a32ad176abfdf47fbec69c3a2e749b974e34e82d"}
Oct 30 23:04:38.434961 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.434938    5503 scope.go:122] "RemoveContainer" containerID="c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d"
Oct 30 23:04:38.444538 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.444522    5503 scope.go:122] "RemoveContainer" containerID="c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d"
Oct 30 23:04:38.444821 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:04:38.444801    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d\": container with ID starting with c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d not found: ID does not exist" containerID="c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d"
Oct 30 23:04:38.444890 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.444829    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d"} err="failed to get container status \"c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d\": rpc error: code = NotFound desc = could not find container \"c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d\": container with ID starting with c2700ba7d34cfcc7c7a763e82f606739602339d2257a7911d318ab4b6452798d not found: ID does not exist"
Oct 30 23:04:38.458369 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.458350    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"]
Oct 30 23:04:38.465956 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:38.465939    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-733055c7/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schel7jq6"]
Oct 30 23:04:39.855127 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:39.855097    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" path="/var/lib/kubelet/pods/4c6f2df0-c745-434f-b256-f6c971020ec3/volumes"
Oct 30 23:04:45.388390 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:04:45.388351    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:04:51.016846 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:51.016816    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:04:51.780464 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:04:51.780415    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:05:00.921341 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:05:00.921310    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:05:15.388783 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:05:15.388746    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:05:23.585555 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:05:23.585524    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:05:23.695337 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:05:23.695304    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:05:45.389125 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:05:45.389089    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:05:51.259038 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:05:51.259001    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:00.995699 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:00.995649    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:01.114854 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:01.114818    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:13.884140 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:13.884112    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:14.006134 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:14.006105    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:15.389770 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:06:15.389727    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:06:20.499462 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:20.499429    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:24.978119 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:24.978087    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:25.077942 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:25.077914    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:25.550352 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:25.550320    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:26.550264 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:26.550224    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:26.649833 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:06:26.649805    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:06:45.388778 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:06:45.388742    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:07:01.098777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:07:01.098741    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:07:01.217356 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:07:01.217328    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:07:01.351925 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:07:01.351837    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:07:15.388524 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:07:15.388489    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:07:33.675873 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:07:33.675791    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:07:33.777404 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:07:33.777367    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:07:45.391364 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:07:45.391322    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:08:01.108916 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:01.108878    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:01.377958 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:01.377862    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:07.000508 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:07.000468    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:09.105401 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:09.105363    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"]
Oct 30 23:08:09.105991 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:09.105592    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy" containerID="cri-o://d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e" gracePeriod=30
Oct 30 23:08:09.109157 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:09.109124    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"]
Oct 30 23:08:09.109390 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:09.109342    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy" containerID="cri-o://4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4" gracePeriod=30
Oct 30 23:08:09.110551 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:09.110521    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"]
Oct 30 23:08:09.110824 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:09.110769    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy" containerID="cri-o://5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6" gracePeriod=30
Oct 30 23:08:09.112185 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:09.112160    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"]
Oct 30 23:08:09.112383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:09.112355    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy" containerID="cri-o://c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997" gracePeriod=30
Oct 30 23:08:13.110058 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:13.110017    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-4wfkx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 23:08:13.110469 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:13.110075    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 23:08:13.841897 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:13.841853    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5k845 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 23:08:13.842072 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:13.841922    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 23:08:14.380475 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.380394    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 23:08:14.422272 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.421738    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-data\") pod \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") "
Oct 30 23:08:14.422272 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.421822    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-token\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-token\") pod \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") "
Oct 30 23:08:14.422272 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.421854    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-podinfo\") pod \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") "
Oct 30 23:08:14.422272 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.421898    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-socket\") pod \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") "
Oct 30 23:08:14.422272 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.421938    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-kube-api-access-n7vvz\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-kube-api-access-n7vvz\") pod \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") "
Oct 30 23:08:14.422272 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.421968    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istiod-ca-cert\") pod \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") "
Oct 30 23:08:14.422272 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.422046    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-envoy\") pod \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") "
Oct 30 23:08:14.422272 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.422043    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-data" pod "c2fe0161-5b10-4d97-97f4-af020ffc5d78" (UID: "c2fe0161-5b10-4d97-97f4-af020ffc5d78"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.423379 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.422408    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-socket" pod "c2fe0161-5b10-4d97-97f4-af020ffc5d78" (UID: "c2fe0161-5b10-4d97-97f4-af020ffc5d78"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.423379 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.422536    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-credential-socket\") pod \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") "
Oct 30 23:08:14.423379 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.422588    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-certs\") pod \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\" (UID: \"c2fe0161-5b10-4d97-97f4-af020ffc5d78\") "
Oct 30 23:08:14.423379 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.422996    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.423379 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.423014    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.424214 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.423706    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-credential-socket" pod "c2fe0161-5b10-4d97-97f4-af020ffc5d78" (UID: "c2fe0161-5b10-4d97-97f4-af020ffc5d78"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.424214 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.423989    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-certs" pod "c2fe0161-5b10-4d97-97f4-af020ffc5d78" (UID: "c2fe0161-5b10-4d97-97f4-af020ffc5d78"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.424538 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.424509    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istiod-ca-cert" pod "c2fe0161-5b10-4d97-97f4-af020ffc5d78" (UID: "c2fe0161-5b10-4d97-97f4-af020ffc5d78"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:08:14.426130 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.426103    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-kube-api-access-n7vvz" pod "c2fe0161-5b10-4d97-97f4-af020ffc5d78" (UID: "c2fe0161-5b10-4d97-97f4-af020ffc5d78"). InnerVolumeSpecName "kube-api-access-n7vvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:14.426543 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.426519    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-envoy" pod "c2fe0161-5b10-4d97-97f4-af020ffc5d78" (UID: "c2fe0161-5b10-4d97-97f4-af020ffc5d78"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.427956 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.427926    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-podinfo" pod "c2fe0161-5b10-4d97-97f4-af020ffc5d78" (UID: "c2fe0161-5b10-4d97-97f4-af020ffc5d78"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:08:14.428154 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.428133    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-token" pod "c2fe0161-5b10-4d97-97f4-af020ffc5d78" (UID: "c2fe0161-5b10-4d97-97f4-af020ffc5d78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:14.438718 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.438660    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 23:08:14.446811 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.446783    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 23:08:14.450287 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.450264    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 23:08:14.492205 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.491632    5503 generic.go:356] "Generic (PLEG): container finished" podID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerID="d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e" exitCode=0
Oct 30 23:08:14.492205 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.491797    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" event={"ID":"c2fe0161-5b10-4d97-97f4-af020ffc5d78","Type":"ContainerDied","Data":"d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e"}
Oct 30 23:08:14.492205 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.491828    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"
Oct 30 23:08:14.492205 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.491848    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx" event={"ID":"c2fe0161-5b10-4d97-97f4-af020ffc5d78","Type":"ContainerDied","Data":"b65d830782be35c248918d59a2da13bad4d7df2b555423ce29f285649668b3df"}
Oct 30 23:08:14.492205 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.491876    5503 scope.go:122] "RemoveContainer" containerID="d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e"
Oct 30 23:08:14.496963 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.496928    5503 generic.go:356] "Generic (PLEG): container finished" podID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerID="c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997" exitCode=0
Oct 30 23:08:14.498306 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.497038    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" event={"ID":"21edf88e-ee91-473c-830d-274ce2f3f1d5","Type":"ContainerDied","Data":"c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997"}
Oct 30 23:08:14.498306 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.497114    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845" event={"ID":"21edf88e-ee91-473c-830d-274ce2f3f1d5","Type":"ContainerDied","Data":"4bbbbfdd800401d0834bb54afd7050fef8c9d80ee9291a792069479de0b1c772"}
Oct 30 23:08:14.498306 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.497044    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"
Oct 30 23:08:14.499097 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.499072    5503 generic.go:356] "Generic (PLEG): container finished" podID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerID="4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4" exitCode=0
Oct 30 23:08:14.499209 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.499170    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" event={"ID":"bd057bcc-31cc-4a62-9093-47fe2363ac11","Type":"ContainerDied","Data":"4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4"}
Oct 30 23:08:14.499268 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.499229    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm" event={"ID":"bd057bcc-31cc-4a62-9093-47fe2363ac11","Type":"ContainerDied","Data":"171a36afae649262061fe42a84ccd8a8f9546a6f954f1520c569985aa216f0ec"}
Oct 30 23:08:14.499268 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.499183    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"
Oct 30 23:08:14.500834 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.500804    5503 generic.go:356] "Generic (PLEG): container finished" podID="e55bda32-45f0-405e-be27-66b3eeefd980" containerID="5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6" exitCode=0
Oct 30 23:08:14.500969 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.500878    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"
Oct 30 23:08:14.501025 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.500882    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" event={"ID":"e55bda32-45f0-405e-be27-66b3eeefd980","Type":"ContainerDied","Data":"5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6"}
Oct 30 23:08:14.501065 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.501013    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h" event={"ID":"e55bda32-45f0-405e-be27-66b3eeefd980","Type":"ContainerDied","Data":"f895a673437d385fcc1596789c52514d73dfeb3e88298e52e8062689dd71e190"}
Oct 30 23:08:14.505828 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.505807    5503 scope.go:122] "RemoveContainer" containerID="e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f"
Oct 30 23:08:14.521929 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.521902    5503 scope.go:122] "RemoveContainer" containerID="d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e"
Oct 30 23:08:14.522364 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:08:14.522338    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e\": container with ID starting with d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e not found: ID does not exist" containerID="d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e"
Oct 30 23:08:14.522458 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.522378    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e"} err="failed to get container status \"d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e\": rpc error: code = NotFound desc = could not find container \"d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e\": container with ID starting with d6e5ba967ff703708ac3f6f921eaa4582ec6a14f56b2fe2470f298efc8eab28e not found: ID does not exist"
Oct 30 23:08:14.522458 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.522404    5503 scope.go:122] "RemoveContainer" containerID="e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f"
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:08:14.522767    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f\": container with ID starting with e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f not found: ID does not exist" containerID="e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f"
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.522805    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f"} err="failed to get container status \"e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f\": rpc error: code = NotFound desc = could not find container \"e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f\": container with ID starting with e4c3bed1b69f26424483c5b45f5e7b165b62e1eaf3efb4964c8ee265c685477f not found: ID does not exist"
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.522831    5503 scope.go:122] "RemoveContainer" containerID="c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997"
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523406    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-kube-api-access-4zdnb\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-kube-api-access-4zdnb\") pod \"bd057bcc-31cc-4a62-9093-47fe2363ac11\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523442    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-credential-socket\") pod \"bd057bcc-31cc-4a62-9093-47fe2363ac11\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523472    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-socket\") pod \"21edf88e-ee91-473c-830d-274ce2f3f1d5\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523521    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-envoy\") pod \"21edf88e-ee91-473c-830d-274ce2f3f1d5\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523554    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-token\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-token\") pod \"21edf88e-ee91-473c-830d-274ce2f3f1d5\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523581    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-certs\") pod \"21edf88e-ee91-473c-830d-274ce2f3f1d5\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523610    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-data\") pod \"e55bda32-45f0-405e-be27-66b3eeefd980\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523642    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e55bda32-45f0-405e-be27-66b3eeefd980-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55bda32-45f0-405e-be27-66b3eeefd980-istiod-ca-cert\") pod \"e55bda32-45f0-405e-be27-66b3eeefd980\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523715    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-data\") pod \"21edf88e-ee91-473c-830d-274ce2f3f1d5\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523757    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-token\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-token\") pod \"bd057bcc-31cc-4a62-9093-47fe2363ac11\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523795    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-credential-socket\") pod \"21edf88e-ee91-473c-830d-274ce2f3f1d5\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.523823    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-certs\") pod \"bd057bcc-31cc-4a62-9093-47fe2363ac11\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") "
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.524071    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-certs" pod "bd057bcc-31cc-4a62-9093-47fe2363ac11" (UID: "bd057bcc-31cc-4a62-9093-47fe2363ac11"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.524119    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-certs" pod "21edf88e-ee91-473c-830d-274ce2f3f1d5" (UID: "21edf88e-ee91-473c-830d-274ce2f3f1d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.524138    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-credential-socket" pod "bd057bcc-31cc-4a62-9093-47fe2363ac11" (UID: "bd057bcc-31cc-4a62-9093-47fe2363ac11"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.524184    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e55bda32-45f0-405e-be27-66b3eeefd980-istiod-ca-cert" pod "e55bda32-45f0-405e-be27-66b3eeefd980" (UID: "e55bda32-45f0-405e-be27-66b3eeefd980"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:08:14.524383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.524296    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-data" pod "21edf88e-ee91-473c-830d-274ce2f3f1d5" (UID: "21edf88e-ee91-473c-830d-274ce2f3f1d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.525656 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.524565    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-data" pod "e55bda32-45f0-405e-be27-66b3eeefd980" (UID: "e55bda32-45f0-405e-be27-66b3eeefd980"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.525656 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.525023    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-socket" pod "21edf88e-ee91-473c-830d-274ce2f3f1d5" (UID: "21edf88e-ee91-473c-830d-274ce2f3f1d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.525656 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.524720    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-credential-socket" pod "21edf88e-ee91-473c-830d-274ce2f3f1d5" (UID: "21edf88e-ee91-473c-830d-274ce2f3f1d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.528151 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.528119    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-credential-socket" pod "e55bda32-45f0-405e-be27-66b3eeefd980" (UID: "e55bda32-45f0-405e-be27-66b3eeefd980"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.528418 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.528293    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-token" pod "21edf88e-ee91-473c-830d-274ce2f3f1d5" (UID: "21edf88e-ee91-473c-830d-274ce2f3f1d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:14.528627 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.528437    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-token" pod "bd057bcc-31cc-4a62-9093-47fe2363ac11" (UID: "bd057bcc-31cc-4a62-9093-47fe2363ac11"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:14.528752 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.528526    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-kube-api-access-4zdnb" pod "bd057bcc-31cc-4a62-9093-47fe2363ac11" (UID: "bd057bcc-31cc-4a62-9093-47fe2363ac11"). InnerVolumeSpecName "kube-api-access-4zdnb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:14.528843 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.528339    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-credential-socket\") pod \"e55bda32-45f0-405e-be27-66b3eeefd980\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") "
Oct 30 23:08:14.528981 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.528925    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-podinfo\") pod \"bd057bcc-31cc-4a62-9093-47fe2363ac11\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") "
Oct 30 23:08:14.529054 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.528995    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-certs\") pod \"e55bda32-45f0-405e-be27-66b3eeefd980\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") "
Oct 30 23:08:14.529110 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.529060    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-podinfo\") pod \"21edf88e-ee91-473c-830d-274ce2f3f1d5\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") "
Oct 30 23:08:14.529169 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.529105    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-kube-api-access-gskp5\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-kube-api-access-gskp5\") pod \"e55bda32-45f0-405e-be27-66b3eeefd980\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") "
Oct 30 23:08:14.529995 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.529914    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-envoy" pod "21edf88e-ee91-473c-830d-274ce2f3f1d5" (UID: "21edf88e-ee91-473c-830d-274ce2f3f1d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.530294 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530270    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-certs" pod "e55bda32-45f0-405e-be27-66b3eeefd980" (UID: "e55bda32-45f0-405e-be27-66b3eeefd980"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.530416 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530398    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-istio-token\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-istio-token\") pod \"e55bda32-45f0-405e-be27-66b3eeefd980\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") "
Oct 30 23:08:14.530483 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530463    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-data\") pod \"bd057bcc-31cc-4a62-9093-47fe2363ac11\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") "
Oct 30 23:08:14.530541 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530502    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-socket\") pod \"bd057bcc-31cc-4a62-9093-47fe2363ac11\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") "
Oct 30 23:08:14.530594 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530537    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-kube-api-access-prwws\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-kube-api-access-prwws\") pod \"21edf88e-ee91-473c-830d-274ce2f3f1d5\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") "
Oct 30 23:08:14.530594 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530577    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/21edf88e-ee91-473c-830d-274ce2f3f1d5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21edf88e-ee91-473c-830d-274ce2f3f1d5-istiod-ca-cert\") pod \"21edf88e-ee91-473c-830d-274ce2f3f1d5\" (UID: \"21edf88e-ee91-473c-830d-274ce2f3f1d5\") "
Oct 30 23:08:14.530718 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530620    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-envoy\") pod \"bd057bcc-31cc-4a62-9093-47fe2363ac11\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") "
Oct 30 23:08:14.530718 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530704    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd057bcc-31cc-4a62-9093-47fe2363ac11-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd057bcc-31cc-4a62-9093-47fe2363ac11-istiod-ca-cert\") pod \"bd057bcc-31cc-4a62-9093-47fe2363ac11\" (UID: \"bd057bcc-31cc-4a62-9093-47fe2363ac11\") "
Oct 30 23:08:14.530827 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530743    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e55bda32-45f0-405e-be27-66b3eeefd980-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55bda32-45f0-405e-be27-66b3eeefd980-istio-podinfo\") pod \"e55bda32-45f0-405e-be27-66b3eeefd980\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") "
Oct 30 23:08:14.530827 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530787    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-socket\") pod \"e55bda32-45f0-405e-be27-66b3eeefd980\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") "
Oct 30 23:08:14.530827 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.530819    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-envoy\") pod \"e55bda32-45f0-405e-be27-66b3eeefd980\" (UID: \"e55bda32-45f0-405e-be27-66b3eeefd980\") "
Oct 30 23:08:14.531516 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531491    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zdnb\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-kube-api-access-4zdnb\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531592 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531520    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531592 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531541    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531592 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531565    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531592 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531578    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531592 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531590    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531607    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531620    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531634    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531649    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7vvz\" (UniqueName: \"kubernetes.io/projected/c2fe0161-5b10-4d97-97f4-af020ffc5d78-kube-api-access-n7vvz\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531784    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e55bda32-45f0-405e-be27-66b3eeefd980-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531799    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531812    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531825    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531842    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/21edf88e-ee91-473c-830d-274ce2f3f1d5-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531855    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531869    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531883    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531907    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531924    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.531941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.531938    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c2fe0161-5b10-4d97-97f4-af020ffc5d78-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.532693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.532191    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-data" pod "bd057bcc-31cc-4a62-9093-47fe2363ac11" (UID: "bd057bcc-31cc-4a62-9093-47fe2363ac11"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.532693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.532520    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-kube-api-access-gskp5" pod "e55bda32-45f0-405e-be27-66b3eeefd980" (UID: "e55bda32-45f0-405e-be27-66b3eeefd980"). InnerVolumeSpecName "kube-api-access-gskp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:14.532806 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.532748    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-socket" pod "bd057bcc-31cc-4a62-9093-47fe2363ac11" (UID: "bd057bcc-31cc-4a62-9093-47fe2363ac11"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.533469 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.533028    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21edf88e-ee91-473c-830d-274ce2f3f1d5-istiod-ca-cert" pod "21edf88e-ee91-473c-830d-274ce2f3f1d5" (UID: "21edf88e-ee91-473c-830d-274ce2f3f1d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:08:14.533469 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.533111    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd057bcc-31cc-4a62-9093-47fe2363ac11-istiod-ca-cert" pod "bd057bcc-31cc-4a62-9093-47fe2363ac11" (UID: "bd057bcc-31cc-4a62-9093-47fe2363ac11"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:08:14.533865 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.533716    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-socket" pod "e55bda32-45f0-405e-be27-66b3eeefd980" (UID: "e55bda32-45f0-405e-be27-66b3eeefd980"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.535317 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.534439    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-envoy" pod "e55bda32-45f0-405e-be27-66b3eeefd980" (UID: "e55bda32-45f0-405e-be27-66b3eeefd980"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.535427 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.535403    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e55bda32-45f0-405e-be27-66b3eeefd980-istio-podinfo" pod "e55bda32-45f0-405e-be27-66b3eeefd980" (UID: "e55bda32-45f0-405e-be27-66b3eeefd980"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:08:14.535916 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.535604    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-istio-token" pod "e55bda32-45f0-405e-be27-66b3eeefd980" (UID: "e55bda32-45f0-405e-be27-66b3eeefd980"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:14.537807 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.537774    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-envoy" pod "bd057bcc-31cc-4a62-9093-47fe2363ac11" (UID: "bd057bcc-31cc-4a62-9093-47fe2363ac11"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:08:14.538917 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.538896    5503 scope.go:122] "RemoveContainer" containerID="c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997"
Oct 30 23:08:14.539345 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:08:14.539319    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997\": container with ID starting with c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997 not found: ID does not exist" containerID="c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997"
Oct 30 23:08:14.539410 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.539361    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997"} err="failed to get container status \"c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997\": rpc error: code = NotFound desc = could not find container \"c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997\": container with ID starting with c3a7713437d98dfc651c5ba8d38016e434940c035af6785680d17ea11700b997 not found: ID does not exist"
Oct 30 23:08:14.539410 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.539383    5503 scope.go:122] "RemoveContainer" containerID="4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4"
Oct 30 23:08:14.540450 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.540413    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-podinfo" pod "bd057bcc-31cc-4a62-9093-47fe2363ac11" (UID: "bd057bcc-31cc-4a62-9093-47fe2363ac11"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:08:14.544103 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.544066    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-podinfo" pod "21edf88e-ee91-473c-830d-274ce2f3f1d5" (UID: "21edf88e-ee91-473c-830d-274ce2f3f1d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:08:14.546751 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.545554    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-kube-api-access-prwws" pod "21edf88e-ee91-473c-830d-274ce2f3f1d5" (UID: "21edf88e-ee91-473c-830d-274ce2f3f1d5"). InnerVolumeSpecName "kube-api-access-prwws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:08:14.550752 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.550721    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"]
Oct 30 23:08:14.553239 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.553213    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-4wfkx"]
Oct 30 23:08:14.555327 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.555311    5503 scope.go:122] "RemoveContainer" containerID="4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4"
Oct 30 23:08:14.555634 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:08:14.555613    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4\": container with ID starting with 4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4 not found: ID does not exist" containerID="4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4"
Oct 30 23:08:14.555717 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.555642    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4"} err="failed to get container status \"4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4\": rpc error: code = NotFound desc = could not find container \"4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4\": container with ID starting with 4f92bdf9f91b2539883cead2f4499b129bd37f8666f59f27421b9e0f8e8427c4 not found: ID does not exist"
Oct 30 23:08:14.555717 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.555656    5503 scope.go:122] "RemoveContainer" containerID="5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6"
Oct 30 23:08:14.564463 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.564441    5503 scope.go:122] "RemoveContainer" containerID="5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6"
Oct 30 23:08:14.564790 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:08:14.564768    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6\": container with ID starting with 5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6 not found: ID does not exist" containerID="5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6"
Oct 30 23:08:14.564872 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.564797    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6"} err="failed to get container status \"5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6\": rpc error: code = NotFound desc = could not find container \"5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6\": container with ID starting with 5e32d3ea07841de82807eae1385ffae2c1d0bcf8520717d8e96dbe0526724ae6 not found: ID does not exist"
Oct 30 23:08:14.632710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632597    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.632710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632629    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/21edf88e-ee91-473c-830d-274ce2f3f1d5-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.632710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632642    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gskp5\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-kube-api-access-gskp5\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.632710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632652    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e55bda32-45f0-405e-be27-66b3eeefd980-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.632710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632678    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.632710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632691    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.632710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632701    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prwws\" (UniqueName: \"kubernetes.io/projected/21edf88e-ee91-473c-830d-274ce2f3f1d5-kube-api-access-prwws\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.632710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632713    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/21edf88e-ee91-473c-830d-274ce2f3f1d5-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.633063 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632723    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd057bcc-31cc-4a62-9093-47fe2363ac11-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.633063 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632731    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd057bcc-31cc-4a62-9093-47fe2363ac11-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.633063 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632739    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e55bda32-45f0-405e-be27-66b3eeefd980-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.633063 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632747    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.633063 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.632755    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e55bda32-45f0-405e-be27-66b3eeefd980-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:08:14.831379 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.831339    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"]
Oct 30 23:08:14.835860 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.835831    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5k845"]
Oct 30 23:08:14.851288 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.851254    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"]
Oct 30 23:08:14.863292 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.863263    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fdlwm"]
Oct 30 23:08:14.882640 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.882606    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"]
Oct 30 23:08:14.897570 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:14.897488    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-sf89h"]
Oct 30 23:08:15.064081 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:15.064051    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:15.197139 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:15.197060    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:15.389824 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:08:15.389785    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:08:15.854797 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:15.854766    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" path="/var/lib/kubelet/pods/21edf88e-ee91-473c-830d-274ce2f3f1d5/volumes"
Oct 30 23:08:15.855197 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:15.855184    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" path="/var/lib/kubelet/pods/bd057bcc-31cc-4a62-9093-47fe2363ac11/volumes"
Oct 30 23:08:15.855549 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:15.855538    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" path="/var/lib/kubelet/pods/c2fe0161-5b10-4d97-97f4-af020ffc5d78/volumes"
Oct 30 23:08:15.856139 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:15.856121    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" path="/var/lib/kubelet/pods/e55bda32-45f0-405e-be27-66b3eeefd980/volumes"
Oct 30 23:08:16.668856 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.668823    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:16.886231 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886181    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"]
Oct 30 23:08:16.886747 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886727    5503 cpu_manager.go:414] "containerMap: removing container" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy"
Oct 30 23:08:16.886862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886748    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy"
Oct 30 23:08:16.886862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886774    5503 cpu_manager.go:414] "containerMap: removing container" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy"
Oct 30 23:08:16.886862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886785    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy"
Oct 30 23:08:16.886862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886795    5503 cpu_manager.go:414] "containerMap: removing container" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerName="main"
Oct 30 23:08:16.886862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886803    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerName="main"
Oct 30 23:08:16.886862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886833    5503 cpu_manager.go:414] "containerMap: removing container" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy"
Oct 30 23:08:16.886862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886841    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy"
Oct 30 23:08:16.886862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886855    5503 cpu_manager.go:414] "containerMap: removing container" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy"
Oct 30 23:08:16.886862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886861    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy"
Oct 30 23:08:16.887378 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886870    5503 cpu_manager.go:414] "containerMap: removing container" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy"
Oct 30 23:08:16.887378 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886877    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy"
Oct 30 23:08:16.887378 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886982    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy"
Oct 30 23:08:16.887378 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.886996    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2fe0161-5b10-4d97-97f4-af020ffc5d78" containerName="istio-proxy"
Oct 30 23:08:16.887378 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.887005    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="e55bda32-45f0-405e-be27-66b3eeefd980" containerName="istio-proxy"
Oct 30 23:08:16.887378 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.887016    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd057bcc-31cc-4a62-9093-47fe2363ac11" containerName="istio-proxy"
Oct 30 23:08:16.887378 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.887027    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="21edf88e-ee91-473c-830d-274ce2f3f1d5" containerName="istio-proxy"
Oct 30 23:08:16.887378 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.887036    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c6f2df0-c745-434f-b256-f6c971020ec3" containerName="main"
Oct 30 23:08:16.891802 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.891781    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:16.900423 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.900396    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"openshift-service-ca.crt\""
Oct 30 23:08:16.900763 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.900548    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-npmrs\""
Oct 30 23:08:16.901044 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.901026    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"kube-root-ca.crt\""
Oct 30 23:08:16.901173 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.901067    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"default-dockercfg-8x4bx\""
Oct 30 23:08:16.902233 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.902211    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"]
Oct 30 23:08:16.905891 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.905868    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-kserve-self-signed-certs\""
Oct 30 23:08:16.952259 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.952161    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf6x7\" (UniqueName: \"kubernetes.io/projected/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-kube-api-access-bf6x7\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c\" (UID: \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:16.952259 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.952207    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c\" (UID: \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:16.964237 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:16.964184    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:17.053573 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:17.053517    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf6x7\" (UniqueName: \"kubernetes.io/projected/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-kube-api-access-bf6x7\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c\" (UID: \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:17.053807 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:17.053608    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c\" (UID: \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:17.056870 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:17.056844    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c\" (UID: \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:17.108248 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:17.108217    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf6x7\" (UniqueName: \"kubernetes.io/projected/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-kube-api-access-bf6x7\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c\" (UID: \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:17.207769 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:17.207684    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:17.510789 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:17.510704    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"]
Oct 30 23:08:17.517229 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:08:17.517196    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8497c27_b38a_4323_9c6f_3d9f1e18c2ed.slice/crio-b7e8a10d1be4fdde82cdd1d5a260f4046ba008bf55894bebc1be4b2ea0c1600f WatchSource:0}: Error finding container b7e8a10d1be4fdde82cdd1d5a260f4046ba008bf55894bebc1be4b2ea0c1600f: Status 404 returned error can't find the container with id b7e8a10d1be4fdde82cdd1d5a260f4046ba008bf55894bebc1be4b2ea0c1600f
Oct 30 23:08:18.527518 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:18.527477    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" event={"ID":"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed","Type":"ContainerStarted","Data":"658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c"}
Oct 30 23:08:18.527974 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:18.527532    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" event={"ID":"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed","Type":"ContainerStarted","Data":"b7e8a10d1be4fdde82cdd1d5a260f4046ba008bf55894bebc1be4b2ea0c1600f"}
Oct 30 23:08:18.527974 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:18.527640    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:18.550167 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:18.550101    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" podStartSLOduration=2.550087491 podStartE2EDuration="2.550087491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:08:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:08:18.548736042 +0000 UTC m=+2940.817817565" watchObservedRunningTime="2026-10-30 23:08:18.550087491 +0000 UTC m=+2940.819169024"
Oct 30 23:08:19.941487 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:19.941454    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:21.683070 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:21.683035    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:27.837510 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:27.837479    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:27.998614 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:27.998577    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:28.107217 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.107180    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:28.183252 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.183211    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"]
Oct 30 23:08:28.189119 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.189086    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:08:28.199643 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.199507    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"default-dockercfg-glldn\""
Oct 30 23:08:28.199643 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.199632    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"openshift-service-ca.crt\""
Oct 30 23:08:28.199862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.199840    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Oct 30 23:08:28.199942 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.199927    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"kube-root-ca.crt\""
Oct 30 23:08:28.200070 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.200055    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-epp-sa-dockercfg-5h7kj\""
Oct 30 23:08:28.200306 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.200276    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"]
Oct 30 23:08:28.263327 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.263289    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:28.360829 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.360740    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c90ff91-eb0a-4cba-baef-8fa61306d01c-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f\" (UID: \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:08:28.360975 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.360896    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs6qp\" (UniqueName: \"kubernetes.io/projected/6c90ff91-eb0a-4cba-baef-8fa61306d01c-kube-api-access-hs6qp\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f\" (UID: \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:08:28.461857 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.461817    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs6qp\" (UniqueName: \"kubernetes.io/projected/6c90ff91-eb0a-4cba-baef-8fa61306d01c-kube-api-access-hs6qp\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f\" (UID: \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:08:28.462019 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.461879    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c90ff91-eb0a-4cba-baef-8fa61306d01c-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f\" (UID: \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:08:28.464526 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.464503    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c90ff91-eb0a-4cba-baef-8fa61306d01c-tls-certs\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f\" (UID: \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:08:28.475485 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.475455    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs6qp\" (UniqueName: \"kubernetes.io/projected/6c90ff91-eb0a-4cba-baef-8fa61306d01c-kube-api-access-hs6qp\") pod \"custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f\" (UID: \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:08:28.507478 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.507447    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:08:28.787912 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:28.787886    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"]
Oct 30 23:08:28.790511 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:08:28.790476    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c90ff91_eb0a_4cba_baef_8fa61306d01c.slice/crio-f7fcf98173fa78e5fd57c5ce6ff9d0c3005ef206182fc7d58b346b36b944ebd6 WatchSource:0}: Error finding container f7fcf98173fa78e5fd57c5ce6ff9d0c3005ef206182fc7d58b346b36b944ebd6: Status 404 returned error can't find the container with id f7fcf98173fa78e5fd57c5ce6ff9d0c3005ef206182fc7d58b346b36b944ebd6
Oct 30 23:08:29.588825 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:29.588787    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" event={"ID":"6c90ff91-eb0a-4cba-baef-8fa61306d01c","Type":"ContainerStarted","Data":"5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2"}
Oct 30 23:08:29.589241 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:29.588843    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" event={"ID":"6c90ff91-eb0a-4cba-baef-8fa61306d01c","Type":"ContainerStarted","Data":"f7fcf98173fa78e5fd57c5ce6ff9d0c3005ef206182fc7d58b346b36b944ebd6"}
Oct 30 23:08:29.589241 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:29.588877    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:08:29.611358 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:29.611300    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" podStartSLOduration=1.6112811379999998 podStartE2EDuration="1.611281138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:08:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:08:29.609720538 +0000 UTC m=+2951.878802059" watchObservedRunningTime="2026-10-30 23:08:29.611281138 +0000 UTC m=+2951.880362663"
Oct 30 23:08:35.473365 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:35.473328    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:08:45.389325 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:08:45.389289    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:08:49.534619 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:49.534585    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:08:54.161618 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.161578    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"]
Oct 30 23:08:54.165850 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.165827    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.170225 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.170193    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-kxprg\""
Oct 30 23:08:54.177949 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.177923    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"]
Oct 30 23:08:54.180359 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.180331    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9d2x7\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-kube-api-access-9d2x7\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.180498 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.180366    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.180498 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.180387    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.180498 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.180407    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.180498 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.180489    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.180638 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.180556    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2afbb22b-8fa2-4695-b392-3a99853dfd94-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.180723 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.180710    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.180762 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.180745    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.180794 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.180766    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.181537 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.181509    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.183890 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.183863    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"]
Oct 30 23:08:54.198857 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.198826    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"]
Oct 30 23:08:54.202731 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.202696    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"]
Oct 30 23:08:54.202911 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.202780    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.221724 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.221693    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"]
Oct 30 23:08:54.281815 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.281784    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.281815 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.281819    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.282071 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.281841    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.282071 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.281915    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.282071 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.281946    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls9g8\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-kube-api-access-ls9g8\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.282071 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.281972    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.282071 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.281990    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.282071 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282049    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.282369 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282156    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.282369 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282254    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.282369 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282274    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.282369 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282284    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.282539 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282374    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f059a50f-09a7-46bf-8ba8-de3332be620f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.282539 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282410    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.282539 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282472    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.282539 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282498    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.282742 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282544    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.282742 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282562    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9d2x7\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-kube-api-access-9d2x7\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.282742 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282607    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.282742 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282641    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7lzm\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-kube-api-access-m7lzm\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.282881 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282857    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.282935 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282879    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.282935 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282897    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.282935 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282914    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.283118 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282933    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.283118 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.282961    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.283118 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.283057    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2afbb22b-8fa2-4695-b392-3a99853dfd94-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.283118 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.283104    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.283271 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.283138    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.283271 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.283219    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.283335 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.283296    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.283680 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.283645    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2afbb22b-8fa2-4695-b392-3a99853dfd94-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.284411 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.284388    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.285259 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.285242    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.295340 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.295312    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.296020 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.296001    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9d2x7\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-kube-api-access-9d2x7\") pod \"openshift-ai-inference-openshift-default-86b669b864-c7w8w\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.384526 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384476    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.384526 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384522    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384550    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384571    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384603    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f059a50f-09a7-46bf-8ba8-de3332be620f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384630    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384700    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384729    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384756    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384790    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7lzm\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-kube-api-access-m7lzm\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384817    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.384840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384843    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.385338 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384871    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.385338 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384934    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.385338 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.384970    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.385338 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385007    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.385338 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385040    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.385338 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385037    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.385338 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385289    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.385883 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385346    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.385883 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385381    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.385883 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385427    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f059a50f-09a7-46bf-8ba8-de3332be620f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.385883 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385722    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.385883 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385827    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.385883 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385840    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.385883 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.385867    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls9g8\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-kube-api-access-ls9g8\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.386235 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.386090    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.386309 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.386287    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.387544 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.387517    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.387776 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.387758    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.387861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.387792    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.388079 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.388061    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.398514 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.398484    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.398865 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.398846    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7lzm\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-kube-api-access-m7lzm\") pod \"openshift-ai-inference-openshift-default-86b669b864-xfb4p\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.399077 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.399061    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.399122 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.399076    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls9g8\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-kube-api-access-ls9g8\") pod \"openshift-ai-inference-openshift-default-86b669b864-fbb5t\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.480712 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.480604    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:54.496285 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.496259    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:54.517611 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.517580    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:54.766852 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.766828    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"]
Oct 30 23:08:54.769238 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:08:54.769211    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2afbb22b_8fa2_4695_b392_3a99853dfd94.slice/crio-44a4c876f5fd35a44b92aed54efa1c57d72aa1b63610c321f69bd5136a7d1ce0 WatchSource:0}: Error finding container 44a4c876f5fd35a44b92aed54efa1c57d72aa1b63610c321f69bd5136a7d1ce0: Status 404 returned error can't find the container with id 44a4c876f5fd35a44b92aed54efa1c57d72aa1b63610c321f69bd5136a7d1ce0
Oct 30 23:08:54.771520 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.771482    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:08:54.771606 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.771558    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:08:54.771606 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.771585    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:08:54.788721 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.788691    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"]
Oct 30 23:08:54.796924 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:08:54.796898    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf059a50f_09a7_46bf_8ba8_de3332be620f.slice/crio-9ee071d835dedb69c48a8dc7fc0f1d4b0a8699cc3eb3921cb2e10cd07a4c7fad WatchSource:0}: Error finding container 9ee071d835dedb69c48a8dc7fc0f1d4b0a8699cc3eb3921cb2e10cd07a4c7fad: Status 404 returned error can't find the container with id 9ee071d835dedb69c48a8dc7fc0f1d4b0a8699cc3eb3921cb2e10cd07a4c7fad
Oct 30 23:08:54.799396 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.799363    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:08:54.799489 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.799429    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:08:54.799489 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.799459    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:08:54.821327 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.821258    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"]
Oct 30 23:08:54.825048 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:08:54.825018    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9026bec_5c53_4ddb_8cf8_c6a92c58ba1f.slice/crio-9fe390b6ae828a2497b71940299635893cbe5ed13990d474d190c56109c7ff72 WatchSource:0}: Error finding container 9fe390b6ae828a2497b71940299635893cbe5ed13990d474d190c56109c7ff72: Status 404 returned error can't find the container with id 9fe390b6ae828a2497b71940299635893cbe5ed13990d474d190c56109c7ff72
Oct 30 23:08:54.827381 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.827341    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:08:54.827481 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.827434    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:08:54.827540 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:54.827479    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:08:55.746693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:55.743175    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" event={"ID":"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f","Type":"ContainerStarted","Data":"ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d"}
Oct 30 23:08:55.769904 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:55.769762    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" event={"ID":"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f","Type":"ContainerStarted","Data":"9fe390b6ae828a2497b71940299635893cbe5ed13990d474d190c56109c7ff72"}
Oct 30 23:08:55.769904 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:55.769809    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" event={"ID":"2afbb22b-8fa2-4695-b392-3a99853dfd94","Type":"ContainerStarted","Data":"24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd"}
Oct 30 23:08:55.769904 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:55.769831    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" event={"ID":"2afbb22b-8fa2-4695-b392-3a99853dfd94","Type":"ContainerStarted","Data":"44a4c876f5fd35a44b92aed54efa1c57d72aa1b63610c321f69bd5136a7d1ce0"}
Oct 30 23:08:55.772964 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:55.772903    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" podStartSLOduration=1.772883328 podStartE2EDuration="1.772883328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:08:55.771480852 +0000 UTC m=+2978.040562375" watchObservedRunningTime="2026-10-30 23:08:55.772883328 +0000 UTC m=+2978.041964852"
Oct 30 23:08:55.775302 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:55.775269    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" event={"ID":"f059a50f-09a7-46bf-8ba8-de3332be620f","Type":"ContainerStarted","Data":"8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4"}
Oct 30 23:08:55.775616 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:55.775588    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" event={"ID":"f059a50f-09a7-46bf-8ba8-de3332be620f","Type":"ContainerStarted","Data":"9ee071d835dedb69c48a8dc7fc0f1d4b0a8699cc3eb3921cb2e10cd07a4c7fad"}
Oct 30 23:08:55.844884 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:55.844806    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" podStartSLOduration=1.8447839209999999 podStartE2EDuration="1.844783921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:08:55.809809953 +0000 UTC m=+2978.078891478" watchObservedRunningTime="2026-10-30 23:08:55.844783921 +0000 UTC m=+2978.113865444"
Oct 30 23:08:56.481753 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:56.481655    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:08:56.483253 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:56.483213    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-c7w8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 23:08:56.483388 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:56.483268    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 23:08:56.502045 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:56.500263    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:08:56.513849 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:56.501981    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fbb5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 23:08:56.514497 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:56.514462    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 23:08:56.520444 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:56.520399    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:08:56.520788 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:56.520751    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-xfb4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 23:08:56.520989 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:56.520951    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 23:08:57.481948 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:57.481908    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-c7w8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 23:08:57.482354 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:57.481967    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 23:08:57.498525 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:57.498492    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fbb5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 23:08:57.498802 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:57.498768    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 23:08:57.522480 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:57.522444    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-xfb4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 23:08:57.522809 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:57.522764    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 23:08:58.481014 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:58.480981    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-c7w8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 23:08:58.481160 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:58.481064    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 23:08:58.503070 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:58.503036    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fbb5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 23:08:58.503628 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:58.503597    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 23:08:58.523899 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:58.523870    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-xfb4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 23:08:58.524161 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:58.524133    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 23:08:59.481429 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:59.481395    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-c7w8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 23:08:59.481713 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:59.481652    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 23:08:59.506693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:59.503185    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fbb5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 23:08:59.506693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:59.503235    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 23:08:59.519698 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:59.518006    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-xfb4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 23:08:59.519698 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:08:59.518053    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 23:09:00.482190 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:00.482145    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-c7w8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused" start-of-body=
Oct 30 23:09:00.482402 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:00.482204    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": dial tcp 10.133.0.72:15021: connect: connection refused"
Oct 30 23:09:00.502689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:00.499117    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fbb5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 23:09:00.502689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:00.499168    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 23:09:00.522692 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:00.519095    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-xfb4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 23:09:00.522692 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:00.519146    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 23:09:00.616931 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:00.616885    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:09:00.649630 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:00.649559    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" podStartSLOduration=6.649538914 podStartE2EDuration="6.649538914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:08:55.846036534 +0000 UTC m=+2978.115118058" watchObservedRunningTime="2026-10-30 23:09:00.649538914 +0000 UTC m=+2982.918620438"
Oct 30 23:09:01.170442 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.170388    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:09:01.484622 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.484553    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:09:01.484947 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.484925    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:09:01.485611 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.485595    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:09:01.500160 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.500140    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:09:01.500412 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.500398    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:09:01.500990 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.500973    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:09:01.521437 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.521417    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:09:01.521712 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.521696    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:09:01.522323 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:01.522305    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:09:06.182729 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:09:06.182699    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:09:15.388325 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:09:15.388291    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:09:45.389852 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:09:45.389812    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:10:01.209381 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:01.209350    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:06.218115 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:06.218083    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:11.476788 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:11.476748    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:15.389769 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:10:15.389729    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:10:18.262143 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:18.262111    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"]
Oct 30 23:10:18.480096 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:18.480059    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:18.643576 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:18.643546    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:19.534656 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:19.534580    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" podUID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:10:26.945652 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:26.945538    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:27.010325 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.010284    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"]
Oct 30 23:10:27.014829 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.014794    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:27.022874 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.022843    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-stop-feature-d9eef2a1\"/\"stop-feature-test-epp-sa-dockercfg-vbt8w\""
Oct 30 23:10:27.025806 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.025778    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"]
Oct 30 23:10:27.061180 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.061144    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:27.132593 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.132551    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef9833b6-b79f-459f-a1dd-c5445571f50d-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg\" (UID: \"ef9833b6-b79f-459f-a1dd-c5445571f50d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:27.132877 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.132689    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgs49\" (UniqueName: \"kubernetes.io/projected/ef9833b6-b79f-459f-a1dd-c5445571f50d-kube-api-access-zgs49\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg\" (UID: \"ef9833b6-b79f-459f-a1dd-c5445571f50d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:27.234059 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.233961    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef9833b6-b79f-459f-a1dd-c5445571f50d-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg\" (UID: \"ef9833b6-b79f-459f-a1dd-c5445571f50d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:27.234228 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.234080    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgs49\" (UniqueName: \"kubernetes.io/projected/ef9833b6-b79f-459f-a1dd-c5445571f50d-kube-api-access-zgs49\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg\" (UID: \"ef9833b6-b79f-459f-a1dd-c5445571f50d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:27.237142 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.237100    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef9833b6-b79f-459f-a1dd-c5445571f50d-tls-certs\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg\" (UID: \"ef9833b6-b79f-459f-a1dd-c5445571f50d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:27.248766 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.248701    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgs49\" (UniqueName: \"kubernetes.io/projected/ef9833b6-b79f-459f-a1dd-c5445571f50d-kube-api-access-zgs49\") pod \"stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg\" (UID: \"ef9833b6-b79f-459f-a1dd-c5445571f50d\") " pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:27.334581 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.334538    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:27.657688 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:27.656623    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"]
Oct 30 23:10:27.661931 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:10:27.661885    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef9833b6_b79f_459f_a1dd_c5445571f50d.slice/crio-6e7c61678b1dd403d4aef54c0b1db56cc9eb62591f14eaa1664fc7f95bbdb486 WatchSource:0}: Error finding container 6e7c61678b1dd403d4aef54c0b1db56cc9eb62591f14eaa1664fc7f95bbdb486: Status 404 returned error can't find the container with id 6e7c61678b1dd403d4aef54c0b1db56cc9eb62591f14eaa1664fc7f95bbdb486
Oct 30 23:10:28.215562 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:28.215523    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" event={"ID":"ef9833b6-b79f-459f-a1dd-c5445571f50d","Type":"ContainerStarted","Data":"18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285"}
Oct 30 23:10:28.216077 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:28.215586    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:28.216077 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:28.215603    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" event={"ID":"ef9833b6-b79f-459f-a1dd-c5445571f50d","Type":"ContainerStarted","Data":"6e7c61678b1dd403d4aef54c0b1db56cc9eb62591f14eaa1664fc7f95bbdb486"}
Oct 30 23:10:28.235845 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:28.235789    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" podStartSLOduration=2.235775597 podStartE2EDuration="2.235775597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:10:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:10:28.235001015 +0000 UTC m=+3070.504082540" watchObservedRunningTime="2026-10-30 23:10:28.235775597 +0000 UTC m=+3070.504857120"
Oct 30 23:10:28.350231 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:28.350194    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:29.104266 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:29.104232    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:29.606127 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:29.606096    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:33.282320 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:33.282275    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" podUID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" containerName="main" containerID="cri-o://658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c" gracePeriod=45
Oct 30 23:10:33.499774 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:33.499729    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"]
Oct 30 23:10:33.537599 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:33.537504    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:33.638050 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:33.638013    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:39.534441 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:39.534400    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" podUID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:10:40.595937 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:40.595899    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" podUID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:10:45.389956 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:10:45.389912    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:10:46.088075 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:46.088029    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:46.189176 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:46.189141    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:10:48.525478 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:48.525436    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" podUID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" containerName="main" containerID="cri-o://5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2" gracePeriod=45
Oct 30 23:10:49.534164 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:49.534128    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" podUID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:10:50.595956 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:50.595921    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" podUID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:10:59.224267 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:59.224228    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:10:59.534514 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:59.534423    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" podUID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:10:59.534724 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:10:59.534544    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:11:00.596146 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:00.596108    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" podUID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:11:00.596539 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:00.596216    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:11:01.264609 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:01.264566    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:03.530909 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:03.530881    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:11:03.672615 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:03.672580    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-kube-api-access-bf6x7\" (UniqueName: \"kubernetes.io/projected/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-kube-api-access-bf6x7\") pod \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\" (UID: \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\") "
Oct 30 23:11:03.672892 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:03.672692    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-tls-certs\" (UniqueName: \"kubernetes.io/secret/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-tls-certs\") pod \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\" (UID: \"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed\") "
Oct 30 23:11:03.675025 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:03.674996    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-tls-certs" pod "f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" (UID: "f8497c27-b38a-4323-9c6f-3d9f1e18c2ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:11:03.675148 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:03.675105    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-kube-api-access-bf6x7" pod "f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" (UID: "f8497c27-b38a-4323-9c6f-3d9f1e18c2ed"). InnerVolumeSpecName "kube-api-access-bf6x7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:11:03.774188 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:03.774137    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:11:03.774188 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:03.774181    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bf6x7\" (UniqueName: \"kubernetes.io/projected/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed-kube-api-access-bf6x7\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:11:04.401352 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:04.401319    5503 generic.go:356] "Generic (PLEG): container finished" podID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" containerID="658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c" exitCode=0
Oct 30 23:11:04.401532 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:04.401365    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"
Oct 30 23:11:04.401532 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:04.401395    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" event={"ID":"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed","Type":"ContainerDied","Data":"658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c"}
Oct 30 23:11:04.401532 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:04.401453    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c" event={"ID":"f8497c27-b38a-4323-9c6f-3d9f1e18c2ed","Type":"ContainerDied","Data":"b7e8a10d1be4fdde82cdd1d5a260f4046ba008bf55894bebc1be4b2ea0c1600f"}
Oct 30 23:11:04.401532 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:04.401481    5503 scope.go:122] "RemoveContainer" containerID="658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c"
Oct 30 23:11:04.411049 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:04.411031    5503 scope.go:122] "RemoveContainer" containerID="658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c"
Oct 30 23:11:04.411350 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:11:04.411331    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c\": container with ID starting with 658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c not found: ID does not exist" containerID="658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c"
Oct 30 23:11:04.411440 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:04.411357    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c"} err="failed to get container status \"658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c\": rpc error: code = NotFound desc = could not find container \"658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c\": container with ID starting with 658f58f12aea11a507acbddb05650d9965b78e25dcb7e92e7a53860b17e4943c not found: ID does not exist"
Oct 30 23:11:04.421072 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:04.421044    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"]
Oct 30 23:11:04.425928 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:04.425902    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-h5x9c"]
Oct 30 23:11:05.855342 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:05.855311    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" path="/var/lib/kubelet/pods/f8497c27-b38a-4323-9c6f-3d9f1e18c2ed/volumes"
Oct 30 23:11:06.308315 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:06.308242    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:10.596454 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:10.596412    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" podUID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:11:11.536603 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:11.536573    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:15.391509 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:11:15.391470    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:11:18.784970 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:18.784932    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:11:18.900150 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:18.900042    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6c90ff91-eb0a-4cba-baef-8fa61306d01c-tls-certs\" (UniqueName: \"kubernetes.io/secret/6c90ff91-eb0a-4cba-baef-8fa61306d01c-tls-certs\") pod \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\" (UID: \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\") "
Oct 30 23:11:18.900336 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:18.900177    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c90ff91-eb0a-4cba-baef-8fa61306d01c-kube-api-access-hs6qp\" (UniqueName: \"kubernetes.io/projected/6c90ff91-eb0a-4cba-baef-8fa61306d01c-kube-api-access-hs6qp\") pod \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\" (UID: \"6c90ff91-eb0a-4cba-baef-8fa61306d01c\") "
Oct 30 23:11:18.902595 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:18.902557    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c90ff91-eb0a-4cba-baef-8fa61306d01c-tls-certs" pod "6c90ff91-eb0a-4cba-baef-8fa61306d01c" (UID: "6c90ff91-eb0a-4cba-baef-8fa61306d01c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:11:18.902755 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:18.902574    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c90ff91-eb0a-4cba-baef-8fa61306d01c-kube-api-access-hs6qp" pod "6c90ff91-eb0a-4cba-baef-8fa61306d01c" (UID: "6c90ff91-eb0a-4cba-baef-8fa61306d01c"). InnerVolumeSpecName "kube-api-access-hs6qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:11:19.001023 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.000979    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c90ff91-eb0a-4cba-baef-8fa61306d01c-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:11:19.001023 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.001014    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs6qp\" (UniqueName: \"kubernetes.io/projected/6c90ff91-eb0a-4cba-baef-8fa61306d01c-kube-api-access-hs6qp\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:11:19.477211 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.477179    5503 generic.go:356] "Generic (PLEG): container finished" podID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" containerID="5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2" exitCode=0
Oct 30 23:11:19.477383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.477227    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"
Oct 30 23:11:19.477383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.477258    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" event={"ID":"6c90ff91-eb0a-4cba-baef-8fa61306d01c","Type":"ContainerDied","Data":"5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2"}
Oct 30 23:11:19.477383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.477315    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f" event={"ID":"6c90ff91-eb0a-4cba-baef-8fa61306d01c","Type":"ContainerDied","Data":"f7fcf98173fa78e5fd57c5ce6ff9d0c3005ef206182fc7d58b346b36b944ebd6"}
Oct 30 23:11:19.477383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.477344    5503 scope.go:122] "RemoveContainer" containerID="5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2"
Oct 30 23:11:19.487005 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.486985    5503 scope.go:122] "RemoveContainer" containerID="5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2"
Oct 30 23:11:19.487292 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:11:19.487273    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2\": container with ID starting with 5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2 not found: ID does not exist" containerID="5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2"
Oct 30 23:11:19.487351 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.487302    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2"} err="failed to get container status \"5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2\": rpc error: code = NotFound desc = could not find container \"5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2\": container with ID starting with 5a0794b137a77200789d050638088d00db51dca5c3650f70745d9eb03209d6a2 not found: ID does not exist"
Oct 30 23:11:19.500960 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.500930    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"]
Oct 30 23:11:19.505797 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.505772    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-router-scheduler-5c6b9d547728f"]
Oct 30 23:11:19.854797 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:19.854766    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" path="/var/lib/kubelet/pods/6c90ff91-eb0a-4cba-baef-8fa61306d01c/volumes"
Oct 30 23:11:23.586448 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:23.586416    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:23.707371 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:23.707332    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:23.812647 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:23.812614    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:24.114101 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:24.114068    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:24.227716 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:24.227651    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:27.144436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:27.144407    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:27.569144 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:27.569045    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:29.972923 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:29.972888    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:35.893776 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:35.893736    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:36.024382 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:36.024350    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:44.568981 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.568946    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:44.821799 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.821710    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"]
Oct 30 23:11:44.822108 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.822095    5503 cpu_manager.go:414] "containerMap: removing container" podUID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" containerName="main"
Oct 30 23:11:44.822156 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.822110    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" containerName="main"
Oct 30 23:11:44.822156 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.822134    5503 cpu_manager.go:414] "containerMap: removing container" podUID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" containerName="main"
Oct 30 23:11:44.822156 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.822139    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" containerName="main"
Oct 30 23:11:44.822263 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.822220    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8497c27-b38a-4323-9c6f-3d9f1e18c2ed" containerName="main"
Oct 30 23:11:44.822263 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.822232    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c90ff91-eb0a-4cba-baef-8fa61306d01c" containerName="main"
Oct 30 23:11:44.826775 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.826755    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:11:44.833579 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.833537    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\""
Oct 30 23:11:44.833863 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.833836    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"default-dockercfg-7h88w\""
Oct 30 23:11:44.833948 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.833839    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-kserve-self-signed-certs\""
Oct 30 23:11:44.833948 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.833870    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\""
Oct 30 23:11:44.834018 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.833969    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-epp-sa-dockercfg-vzjl7\""
Oct 30 23:11:44.853159 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.853130    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"]
Oct 30 23:11:44.921192 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.921157    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:44.950927 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.950895    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqzcm\" (UniqueName: \"kubernetes.io/projected/43bfc701-8eb3-41a4-b01d-402fec5d97a0-kube-api-access-kqzcm\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n\" (UID: \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:11:44.951100 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:44.950959    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43bfc701-8eb3-41a4-b01d-402fec5d97a0-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n\" (UID: \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:11:45.051718 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.051658    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqzcm\" (UniqueName: \"kubernetes.io/projected/43bfc701-8eb3-41a4-b01d-402fec5d97a0-kube-api-access-kqzcm\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n\" (UID: \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:11:45.051890 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.051756    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43bfc701-8eb3-41a4-b01d-402fec5d97a0-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n\" (UID: \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:11:45.054326 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.054296    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43bfc701-8eb3-41a4-b01d-402fec5d97a0-tls-certs\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n\" (UID: \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:11:45.069800 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.069768    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqzcm\" (UniqueName: \"kubernetes.io/projected/43bfc701-8eb3-41a4-b01d-402fec5d97a0-kube-api-access-kqzcm\") pod \"router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n\" (UID: \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:11:45.103052 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.103014    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:11:45.142889 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.142856    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:11:45.389247 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:11:45.389161    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:11:45.428173 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.428149    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"]
Oct 30 23:11:45.609947 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.609911    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" event={"ID":"43bfc701-8eb3-41a4-b01d-402fec5d97a0","Type":"ContainerStarted","Data":"7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0"}
Oct 30 23:11:45.610330 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.609966    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" event={"ID":"43bfc701-8eb3-41a4-b01d-402fec5d97a0","Type":"ContainerStarted","Data":"fa725f6d8946881b527192b3957dbfb04ad20034dec67241c9192ee6f16cd34b"}
Oct 30 23:11:45.610330 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.610027    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:11:45.637685 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:11:45.637128    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" podStartSLOduration=1.637109041 podStartE2EDuration="1.637109041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:11:45.635036059 +0000 UTC m=+3147.904117585" watchObservedRunningTime="2026-10-30 23:11:45.637109041 +0000 UTC m=+3147.906190565"
Oct 30 23:12:01.324088 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:01.324047    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:12:06.354768 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:06.354725    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:12:11.597525 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:11.597491    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:12:15.389045 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:12:15.389003    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:12:16.617552 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:16.617522    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:12:28.462519 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:28.462478    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:12:28.467303 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:28.467276    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"]
Oct 30 23:12:28.602246 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:28.602214    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:12:29.224260 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:29.224229    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:12:37.832830 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:37.832791    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:12:37.931623 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:37.931585    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:12:39.224174 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:39.224136    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:12:43.487360 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:43.487317    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerName="main" containerID="cri-o://18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285" gracePeriod=45
Oct 30 23:12:45.389021 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:12:45.388987    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:12:49.224125 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:49.224087    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:12:49.224922 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:49.224195    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:12:59.223560 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:12:59.223517    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:13:01.377279 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:01.377242    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:07.260468 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:07.260436    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:09.224204 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:09.224170    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:13:11.653612 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:11.653564    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:13.722289 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:13.722264    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:13:13.825653 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:13.825569    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ef9833b6-b79f-459f-a1dd-c5445571f50d-tls-certs\" (UniqueName: \"kubernetes.io/secret/ef9833b6-b79f-459f-a1dd-c5445571f50d-tls-certs\") pod \"ef9833b6-b79f-459f-a1dd-c5445571f50d\" (UID: \"ef9833b6-b79f-459f-a1dd-c5445571f50d\") "
Oct 30 23:13:13.825653 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:13.825648    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef9833b6-b79f-459f-a1dd-c5445571f50d-kube-api-access-zgs49\" (UniqueName: \"kubernetes.io/projected/ef9833b6-b79f-459f-a1dd-c5445571f50d-kube-api-access-zgs49\") pod \"ef9833b6-b79f-459f-a1dd-c5445571f50d\" (UID: \"ef9833b6-b79f-459f-a1dd-c5445571f50d\") "
Oct 30 23:13:13.828168 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:13.828123    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef9833b6-b79f-459f-a1dd-c5445571f50d-tls-certs" pod "ef9833b6-b79f-459f-a1dd-c5445571f50d" (UID: "ef9833b6-b79f-459f-a1dd-c5445571f50d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:13:13.828291 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:13.828222    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef9833b6-b79f-459f-a1dd-c5445571f50d-kube-api-access-zgs49" pod "ef9833b6-b79f-459f-a1dd-c5445571f50d" (UID: "ef9833b6-b79f-459f-a1dd-c5445571f50d"). InnerVolumeSpecName "kube-api-access-zgs49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:13:13.926631 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:13.926586    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ef9833b6-b79f-459f-a1dd-c5445571f50d-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:13:13.926631 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:13.926624    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgs49\" (UniqueName: \"kubernetes.io/projected/ef9833b6-b79f-459f-a1dd-c5445571f50d-kube-api-access-zgs49\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:13:14.051210 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:14.051176    5503 generic.go:356] "Generic (PLEG): container finished" podID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerID="18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285" exitCode=0
Oct 30 23:13:14.051400 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:14.051243    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"
Oct 30 23:13:14.051400 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:14.051258    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" event={"ID":"ef9833b6-b79f-459f-a1dd-c5445571f50d","Type":"ContainerDied","Data":"18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285"}
Oct 30 23:13:14.051400 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:14.051328    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg" event={"ID":"ef9833b6-b79f-459f-a1dd-c5445571f50d","Type":"ContainerDied","Data":"6e7c61678b1dd403d4aef54c0b1db56cc9eb62591f14eaa1664fc7f95bbdb486"}
Oct 30 23:13:14.051400 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:14.051358    5503 scope.go:122] "RemoveContainer" containerID="18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285"
Oct 30 23:13:14.060465 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:14.060447    5503 scope.go:122] "RemoveContainer" containerID="18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285"
Oct 30 23:13:14.060796 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:13:14.060771    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285\": container with ID starting with 18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285 not found: ID does not exist" containerID="18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285"
Oct 30 23:13:14.060863 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:14.060807    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285"} err="failed to get container status \"18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285\": rpc error: code = NotFound desc = could not find container \"18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285\": container with ID starting with 18e60529defeb2b16ac2da2581c5a87688ef092b38bccfe4319caac145e4f285 not found: ID does not exist"
Oct 30 23:13:14.070400 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:14.070366    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"]
Oct 30 23:13:14.072633 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:14.072607    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-stop-feature-d9eef2a1/stop-feature-test-kserve-router-scheduler-6c5444664c-t66wg"]
Oct 30 23:13:15.388686 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:13:15.388628    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:13:15.855793 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:15.855763    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" path="/var/lib/kubelet/pods/ef9833b6-b79f-459f-a1dd-c5445571f50d/volumes"
Oct 30 23:13:18.729994 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:18.729893    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:18.889769 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:18.889737    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:18.993984 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:18.993874    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:23.172676 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:23.172633    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:25.981582 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:25.981546    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:32.037626 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:32.037543    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:32.132904 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:32.132865    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:45.391697 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:13:45.391634    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:13:49.972485 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:49.972449    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"]
Oct 30 23:13:49.994285 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:49.994245    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:50.108963 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:50.108932    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:13:56.617712 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:56.617645    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" podUID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:13:58.515760 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:13:58.515725    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:01.087053 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:01.087019    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:01.429053 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:01.429022    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:01.873980 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:01.873938    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:04.992799 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:04.992762    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" podUID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" containerName="main" containerID="cri-o://7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0" gracePeriod=45
Oct 30 23:14:06.464830 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:06.464791    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:06.617044 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:06.616999    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" podUID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:14:11.704950 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:11.704908    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:15.390038 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:14:15.389999    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:14:16.617191 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:16.617150    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" podUID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:14:16.617657 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:16.617310    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:14:26.617684 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:26.617638    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" podUID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:14:35.242371 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.242347    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:14:35.346794 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.346768    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43bfc701-8eb3-41a4-b01d-402fec5d97a0-kube-api-access-kqzcm\" (UniqueName: \"kubernetes.io/projected/43bfc701-8eb3-41a4-b01d-402fec5d97a0-kube-api-access-kqzcm\") pod \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\" (UID: \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\") "
Oct 30 23:14:35.346925 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.346882    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/43bfc701-8eb3-41a4-b01d-402fec5d97a0-tls-certs\" (UniqueName: \"kubernetes.io/secret/43bfc701-8eb3-41a4-b01d-402fec5d97a0-tls-certs\") pod \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\" (UID: \"43bfc701-8eb3-41a4-b01d-402fec5d97a0\") "
Oct 30 23:14:35.349039 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.349018    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43bfc701-8eb3-41a4-b01d-402fec5d97a0-kube-api-access-kqzcm" pod "43bfc701-8eb3-41a4-b01d-402fec5d97a0" (UID: "43bfc701-8eb3-41a4-b01d-402fec5d97a0"). InnerVolumeSpecName "kube-api-access-kqzcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:14:35.349125 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.349036    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43bfc701-8eb3-41a4-b01d-402fec5d97a0-tls-certs" pod "43bfc701-8eb3-41a4-b01d-402fec5d97a0" (UID: "43bfc701-8eb3-41a4-b01d-402fec5d97a0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:14:35.447600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.447575    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43bfc701-8eb3-41a4-b01d-402fec5d97a0-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:35.447600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.447599    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqzcm\" (UniqueName: \"kubernetes.io/projected/43bfc701-8eb3-41a4-b01d-402fec5d97a0-kube-api-access-kqzcm\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:35.532906 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.532882    5503 generic.go:356] "Generic (PLEG): container finished" podID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" containerID="7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0" exitCode=0
Oct 30 23:14:35.533016 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.532923    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"
Oct 30 23:14:35.533016 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.532946    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" event={"ID":"43bfc701-8eb3-41a4-b01d-402fec5d97a0","Type":"ContainerDied","Data":"7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0"}
Oct 30 23:14:35.533016 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.533001    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n" event={"ID":"43bfc701-8eb3-41a4-b01d-402fec5d97a0","Type":"ContainerDied","Data":"fa725f6d8946881b527192b3957dbfb04ad20034dec67241c9192ee6f16cd34b"}
Oct 30 23:14:35.533185 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.533034    5503 scope.go:122] "RemoveContainer" containerID="7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0"
Oct 30 23:14:35.542189 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.542170    5503 scope.go:122] "RemoveContainer" containerID="7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0"
Oct 30 23:14:35.542437 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:14:35.542420    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0\": container with ID starting with 7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0 not found: ID does not exist" containerID="7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0"
Oct 30 23:14:35.542480 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.542446    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0"} err="failed to get container status \"7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0\": rpc error: code = NotFound desc = could not find container \"7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0\": container with ID starting with 7f2d6a71cef8de4fb22f787fe34e2a588a245aaf6ab6d2e280b4002fbc9082b0 not found: ID does not exist"
Oct 30 23:14:35.554870 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.554846    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"]
Oct 30 23:14:35.559583 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.559559    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-router-scheduler-57f69774b-trr8n"]
Oct 30 23:14:35.854590 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:35.854549    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" path="/var/lib/kubelet/pods/43bfc701-8eb3-41a4-b01d-402fec5d97a0/volumes"
Oct 30 23:14:40.301127 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:40.301092    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:40.403053 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:40.403025    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:40.502883 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:40.502839    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:45.322830 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.322795    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:45.389998 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:14:45.389958    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:14:45.817200 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.817151    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:45.989384 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.989349    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"]
Oct 30 23:14:45.990002 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.989977    5503 cpu_manager.go:414] "containerMap: removing container" podUID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" containerName="main"
Oct 30 23:14:45.990194 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.990168    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" containerName="main"
Oct 30 23:14:45.990358 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.990211    5503 cpu_manager.go:414] "containerMap: removing container" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerName="main"
Oct 30 23:14:45.990358 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.990219    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerName="main"
Oct 30 23:14:45.990486 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.990390    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="43bfc701-8eb3-41a4-b01d-402fec5d97a0" containerName="main"
Oct 30 23:14:45.990486 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.990406    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef9833b6-b79f-459f-a1dd-c5445571f50d" containerName="main"
Oct 30 23:14:45.995249 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:45.995228    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:14:46.005376 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.005334    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"default-dockercfg-pjngx\""
Oct 30 23:14:46.005452 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.005382    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Oct 30 23:14:46.005452 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.005403    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-epp-sa-dockercfg-dqzmg\""
Oct 30 23:14:46.005563 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.005460    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"openshift-service-ca.crt\""
Oct 30 23:14:46.005563 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.005485    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"kube-root-ca.crt\""
Oct 30 23:14:46.011382 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.011357    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"]
Oct 30 23:14:46.028042 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.028010    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4xh7\" (UniqueName: \"kubernetes.io/projected/d3cd7f22-2804-4ecd-b4fe-878da39003bb-kube-api-access-d4xh7\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj\" (UID: \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:14:46.028154 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.028132    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cd7f22-2804-4ecd-b4fe-878da39003bb-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj\" (UID: \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:14:46.050571 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.050542    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:14:46.128653 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.128635    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4xh7\" (UniqueName: \"kubernetes.io/projected/d3cd7f22-2804-4ecd-b4fe-878da39003bb-kube-api-access-d4xh7\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj\" (UID: \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:14:46.128796 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.128746    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cd7f22-2804-4ecd-b4fe-878da39003bb-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj\" (UID: \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:14:46.131295 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.131275    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cd7f22-2804-4ecd-b4fe-878da39003bb-tls-certs\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj\" (UID: \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:14:46.141847 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.141824    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4xh7\" (UniqueName: \"kubernetes.io/projected/d3cd7f22-2804-4ecd-b4fe-878da39003bb-kube-api-access-d4xh7\") pod \"llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj\" (UID: \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:14:46.313560 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.313538    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:14:46.610405 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:46.610327    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"]
Oct 30 23:14:47.601600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:47.601570    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" event={"ID":"d3cd7f22-2804-4ecd-b4fe-878da39003bb","Type":"ContainerStarted","Data":"b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f"}
Oct 30 23:14:47.601802 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:47.601617    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" event={"ID":"d3cd7f22-2804-4ecd-b4fe-878da39003bb","Type":"ContainerStarted","Data":"db056ab4217fc31c0e03796da77efa59f395242a2d2b144b6953c5bdd02c73ca"}
Oct 30 23:14:47.601802 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:47.601707    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:14:47.621526 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:47.621479    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" podStartSLOduration=2.621466888 podStartE2EDuration="2.621466888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:14:47.620742428 +0000 UTC m=+3329.889823953" watchObservedRunningTime="2026-10-30 23:14:47.621466888 +0000 UTC m=+3329.890548411"
Oct 30 23:14:50.236654 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:14:50.236620    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:01.488564 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:01.488530    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:06.511887 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:06.511851    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:07.156486 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:07.156422    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:07.886865 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:07.886816    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:11.767683 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:11.767635    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:13.901759 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:13.901724    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:15.389761 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:15:15.389729    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:15:18.607755 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:18.607730    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:15:24.406178 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:24.406139    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:28.106022 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:28.105988    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:28.968560 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:28.968517    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:29.092023 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:29.091991    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:44.637728 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:44.637693    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:44.787789 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:44.787762    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:44.901556 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:44.901486    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:45.388895 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:15:45.388859    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:15:49.461517 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:49.461477    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:51.077258 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:51.077216    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:15:51.177435 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:15:51.177410    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:16:01.551472 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:16:01.551434    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:16:06.567108 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:16:06.567076    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:16:11.858113 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:16:11.858082    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:16:15.388989 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:16:15.388951    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:16:45.388934 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:16:45.388842    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:16:54.979148 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:16:54.979116    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"]
Oct 30 23:16:54.985727 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:16:54.985694    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:16:55.094737 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:16:55.094688    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:16:58.607748 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:16:58.607711    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:17:01.604051 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:01.603990    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:06.623934 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:06.623902    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:08.608267 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:08.608234    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:17:10.001474 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:10.001419    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerName="main" containerID="cri-o://b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f" gracePeriod=45
Oct 30 23:17:11.886098 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:11.886066    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:15.388793 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:17:15.388749    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:17:18.607625 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:18.607592    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:17:18.608140 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:18.607720    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:17:28.608389 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:28.608356    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:17:38.608070 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:38.608026    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:17:40.240336 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.240309    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:17:40.324407 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.324336    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3cd7f22-2804-4ecd-b4fe-878da39003bb-kube-api-access-d4xh7\" (UniqueName: \"kubernetes.io/projected/d3cd7f22-2804-4ecd-b4fe-878da39003bb-kube-api-access-d4xh7\") pod \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\" (UID: \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\") "
Oct 30 23:17:40.324407 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.324402    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3cd7f22-2804-4ecd-b4fe-878da39003bb-tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cd7f22-2804-4ecd-b4fe-878da39003bb-tls-certs\") pod \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\" (UID: \"d3cd7f22-2804-4ecd-b4fe-878da39003bb\") "
Oct 30 23:17:40.326596 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.326571    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3cd7f22-2804-4ecd-b4fe-878da39003bb-tls-certs" pod "d3cd7f22-2804-4ecd-b4fe-878da39003bb" (UID: "d3cd7f22-2804-4ecd-b4fe-878da39003bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:17:40.326712 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.326605    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3cd7f22-2804-4ecd-b4fe-878da39003bb-kube-api-access-d4xh7" pod "d3cd7f22-2804-4ecd-b4fe-878da39003bb" (UID: "d3cd7f22-2804-4ecd-b4fe-878da39003bb"). InnerVolumeSpecName "kube-api-access-d4xh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:17:40.425039 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.425014    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3cd7f22-2804-4ecd-b4fe-878da39003bb-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:40.425039 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.425039    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4xh7\" (UniqueName: \"kubernetes.io/projected/d3cd7f22-2804-4ecd-b4fe-878da39003bb-kube-api-access-d4xh7\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:40.427197 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.427173    5503 generic.go:356] "Generic (PLEG): container finished" podID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerID="b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f" exitCode=0
Oct 30 23:17:40.427284 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.427221    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" event={"ID":"d3cd7f22-2804-4ecd-b4fe-878da39003bb","Type":"ContainerDied","Data":"b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f"}
Oct 30 23:17:40.427284 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.427271    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"
Oct 30 23:17:40.427284 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.427282    5503 scope.go:122] "RemoveContainer" containerID="b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f"
Oct 30 23:17:40.427381 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.427270    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj" event={"ID":"d3cd7f22-2804-4ecd-b4fe-878da39003bb","Type":"ContainerDied","Data":"db056ab4217fc31c0e03796da77efa59f395242a2d2b144b6953c5bdd02c73ca"}
Oct 30 23:17:40.436988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.436971    5503 scope.go:122] "RemoveContainer" containerID="b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f"
Oct 30 23:17:40.437261 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:17:40.437243    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f\": container with ID starting with b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f not found: ID does not exist" containerID="b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f"
Oct 30 23:17:40.437306 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.437270    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f"} err="failed to get container status \"b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f\": rpc error: code = NotFound desc = could not find container \"b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f\": container with ID starting with b95fb3ff31b8cf3cafa7eb5325035d06ffa18b7368a6cfbb078ccfb0c279c52f not found: ID does not exist"
Oct 30 23:17:40.450365 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.450343    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"]
Oct 30 23:17:40.457347 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:40.457327    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc44d181485fad85e662eb092f3749502f-kserve-router-schemj4dj"]
Oct 30 23:17:41.854769 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:41.854736    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" path="/var/lib/kubelet/pods/d3cd7f22-2804-4ecd-b4fe-878da39003bb/volumes"
Oct 30 23:17:45.291596 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:45.291566    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:45.388621 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:17:45.388586    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:17:45.434758 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:45.434731    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:47.553642 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.553607    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:47.735726 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.735690    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"]
Oct 30 23:17:47.736351 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.736326    5503 cpu_manager.go:414] "containerMap: removing container" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerName="main"
Oct 30 23:17:47.736351 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.736351    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerName="main"
Oct 30 23:17:47.736526 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.736450    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3cd7f22-2804-4ecd-b4fe-878da39003bb" containerName="main"
Oct 30 23:17:47.741171 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.741146    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:17:47.749277 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.749254    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Oct 30 23:17:47.749435 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.749305    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-qbdtm\""
Oct 30 23:17:47.749435 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.749342    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-epp-sa-dockercfg-mf2mw\""
Oct 30 23:17:47.749435 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.749414    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Oct 30 23:17:47.750594 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.750435    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Oct 30 23:17:47.751354 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.751333    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"]
Oct 30 23:17:47.800329 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.800305    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:47.892189 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.892154    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fcpp\" (UniqueName: \"kubernetes.io/projected/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-kube-api-access-6fcpp\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47\" (UID: \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:17:47.892386 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.892224    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47\" (UID: \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:17:47.993239 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.993205    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fcpp\" (UniqueName: \"kubernetes.io/projected/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-kube-api-access-6fcpp\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47\" (UID: \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:17:47.993415 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.993309    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47\" (UID: \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:17:47.995969 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:47.995942    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47\" (UID: \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:17:48.017700 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:48.017637    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fcpp\" (UniqueName: \"kubernetes.io/projected/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-kube-api-access-6fcpp\") pod \"custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47\" (UID: \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:17:48.057580 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:48.057554    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:17:48.353855 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:48.353823    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"]
Oct 30 23:17:48.478787 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:48.478658    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" event={"ID":"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967","Type":"ContainerStarted","Data":"0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c"}
Oct 30 23:17:48.478787 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:48.478746    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" event={"ID":"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967","Type":"ContainerStarted","Data":"bedbcd437e36202daf8034095eecf628b9f389c0eac00245fdf70e81241de826"}
Oct 30 23:17:48.479532 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:48.479506    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:17:48.513408 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:48.513340    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" podStartSLOduration=1.51330554 podStartE2EDuration="1.51330554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:17:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:17:48.512162023 +0000 UTC m=+3510.781243546" watchObservedRunningTime="2026-10-30 23:17:48.51330554 +0000 UTC m=+3510.782387063"
Oct 30 23:17:50.785894 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:50.785858    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:52.792399 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:52.792360    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:58.702776 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:58.702742    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:17:58.805276 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:17:58.805246    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:01.084978 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:01.084948    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:01.656542 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:01.656507    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:06.678119 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:06.678085    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:10.872355 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:10.872318    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:15.388648 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:18:15.388612    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:18:17.190769 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:17.190737    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:17.301649 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:17.301626    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:20.489145 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:20.489119    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:18:21.931835 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:21.931796    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:32.545347 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:32.545311    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:32.647585 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:18:32.647557    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:18:45.388185 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:18:45.388145    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:19:06.767967 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:19:06.767930    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:19:11.719343 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:19:11.719311    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:19:15.388715 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:19:15.388647    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:19:22.028119 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:19:22.028085    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:19:39.223437 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:19:39.223405    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:19:45.388834 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:19:45.388799    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:19:56.512750 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:19:56.512717    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"]
Oct 30 23:19:56.530141 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:19:56.530107    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:19:56.630298 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:19:56.630253    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:00.489433 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:00.489398    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:20:06.797652 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:06.797617    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:10.489490 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:10.489453    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:20:11.535612 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:11.535572    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerName="main" containerID="cri-o://0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c" gracePeriod=45
Oct 30 23:20:15.388410 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:20:15.388372    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:20:20.113256 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:20.113217    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:20.489877 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:20.489803    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:20:20.490022 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:20.489945    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:20:21.698415 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:21.698382    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:22.044229 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:22.044165    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:23.013423 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:23.013393    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:23.108146 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:23.108117    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:23.701469 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:23.701434    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:30.489581 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:30.489548    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:20:35.355131 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:35.355101    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:35.452346 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:35.452318    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:40.488872 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:40.488832    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:20:41.783370 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:41.783344    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:20:41.870844 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:41.870817    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-tls-certs\") pod \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\" (UID: \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\") "
Oct 30 23:20:41.870973 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:41.870927    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-kube-api-access-6fcpp\" (UniqueName: \"kubernetes.io/projected/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-kube-api-access-6fcpp\") pod \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\" (UID: \"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967\") "
Oct 30 23:20:41.873030 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:41.873000    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-tls-certs" pod "8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" (UID: "8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:20:41.873129 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:41.873089    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-kube-api-access-6fcpp" pod "8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" (UID: "8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967"). InnerVolumeSpecName "kube-api-access-6fcpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:20:41.971640 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:41.971618    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fcpp\" (UniqueName: \"kubernetes.io/projected/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-kube-api-access-6fcpp\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:41.971640 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:41.971641    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:42.286680 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.286590    5503 generic.go:356] "Generic (PLEG): container finished" podID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerID="0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c" exitCode=0
Oct 30 23:20:42.286680 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.286634    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"
Oct 30 23:20:42.286680 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.286645    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" event={"ID":"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967","Type":"ContainerDied","Data":"0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c"}
Oct 30 23:20:42.286892 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.286709    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47" event={"ID":"8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967","Type":"ContainerDied","Data":"bedbcd437e36202daf8034095eecf628b9f389c0eac00245fdf70e81241de826"}
Oct 30 23:20:42.286892 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.286732    5503 scope.go:122] "RemoveContainer" containerID="0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c"
Oct 30 23:20:42.296446 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.296428    5503 scope.go:122] "RemoveContainer" containerID="0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c"
Oct 30 23:20:42.296736 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:20:42.296707    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c\": container with ID starting with 0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c not found: ID does not exist" containerID="0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c"
Oct 30 23:20:42.296826 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.296746    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c"} err="failed to get container status \"0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c\": rpc error: code = NotFound desc = could not find container \"0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c\": container with ID starting with 0ad15d473dd466cd237beeb53824b7a98203a7de95bf30bdf6453217f2f16e2c not found: ID does not exist"
Oct 30 23:20:42.309922 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.309899    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"]
Oct 30 23:20:42.317312 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.317290    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-router-scheduler-76c9579w47"]
Oct 30 23:20:42.743136 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:42.743102    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:43.873518 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:43.873477    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" path="/var/lib/kubelet/pods/8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967/volumes"
Oct 30 23:20:45.456693 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:20:45.454887    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:20:46.830755 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:46.830715    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:46.945786 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:46.945754    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:53.508686 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:53.508610    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:55.235478 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.235444    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"]
Oct 30 23:20:55.235987 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.235845    5503 cpu_manager.go:414] "containerMap: removing container" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerName="main"
Oct 30 23:20:55.235987 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.235858    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerName="main"
Oct 30 23:20:55.235987 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.235939    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f2952e2-f10c-4a4b-8e3c-4fa9fa4d3967" containerName="main"
Oct 30 23:20:55.238836 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.238818    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.255195 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.255164    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"]
Oct 30 23:20:55.357074 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.357049    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.357185 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.357079    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.357185 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.357097    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.357185 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.357115    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.357304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.357242    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.357304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.357295    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.357365 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.357349    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq4rl\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-kube-api-access-lq4rl\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.357396 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.357376    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.357431 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.357399    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.457803 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.457781    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.457908 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.457805    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.457908 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.457821    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.457908 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.457838    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.457908 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.457872    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.458104 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.457926    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.458104 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.457953    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq4rl\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-kube-api-access-lq4rl\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.458104 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.457975    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.458104 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.457998    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.458325 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.458278    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.458325 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.458309    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.458430 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.458344    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.458557 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.458537    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.458751 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.458732    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.460140 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.460123    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.460531 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.460515    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.471016 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.470996    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq4rl\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-kube-api-access-lq4rl\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.471108 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.471089    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-7tpn9\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.552877 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.552836    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:55.835423 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.835402    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"]
Oct 30 23:20:55.837723 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:20:55.837697    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda75d6804_7b72_4753_9ae3_0cbe3dadd26f.slice/crio-1cdcd61f6f4e30cadc6839ae2940827bae09b21b578e27fe5ce0bab1fe25bf38 WatchSource:0}: Error finding container 1cdcd61f6f4e30cadc6839ae2940827bae09b21b578e27fe5ce0bab1fe25bf38: Status 404 returned error can't find the container with id 1cdcd61f6f4e30cadc6839ae2940827bae09b21b578e27fe5ce0bab1fe25bf38
Oct 30 23:20:55.839652 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.839624    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:20:55.839742 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.839708    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:20:55.839742 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:55.839738    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:20:56.422814 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:56.422768    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" event={"ID":"a75d6804-7b72-4753-9ae3-0cbe3dadd26f","Type":"ContainerStarted","Data":"43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d"}
Oct 30 23:20:56.423267 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:56.422841    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" event={"ID":"a75d6804-7b72-4753-9ae3-0cbe3dadd26f","Type":"ContainerStarted","Data":"1cdcd61f6f4e30cadc6839ae2940827bae09b21b578e27fe5ce0bab1fe25bf38"}
Oct 30 23:20:56.448113 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:56.448065    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" podStartSLOduration=1.448052406 podStartE2EDuration="1.448052406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:20:56.446933335 +0000 UTC m=+3698.716014859" watchObservedRunningTime="2026-10-30 23:20:56.448052406 +0000 UTC m=+3698.717133930"
Oct 30 23:20:56.554040 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:56.554003    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:20:56.555027 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:56.555006    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-7tpn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 23:20:56.555077 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:56.555050    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 23:20:57.554123 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:57.554091    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-7tpn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 23:20:57.554518 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:57.554139    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 23:20:58.554122 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:58.554088    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-7tpn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 23:20:58.554307 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:58.554136    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 23:20:59.520507 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:59.520460    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:20:59.553509 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:59.553482    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-7tpn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 23:20:59.553608 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:59.553530    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 23:20:59.621843 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:20:59.621801    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:21:00.553226 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:00.553191    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-7tpn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 23:21:00.553374 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:00.553249    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 23:21:01.553355 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:01.553317    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-7tpn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused" start-of-body=
Oct 30 23:21:01.553845 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:01.553373    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.79:15021/healthz/ready\": dial tcp 10.133.0.79:15021: connect: connection refused"
Oct 30 23:21:02.557180 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:02.557150    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:21:02.557738 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:02.557429    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:21:02.558199 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:02.558182    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:21:15.389213 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:21:15.389176    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:21:16.792052 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:16.792019    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:21:16.895721 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:16.895696    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:21:22.093520 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:22.093488    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:21:23.724314 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:21:23.724284    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:21:45.387814 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:21:45.387780    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:22:15.389383 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:22:15.389346    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:22:16.850274 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:16.850236    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:33.696558 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:33.696470    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:42.158562 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:42.158504    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:45.388835 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:22:45.388801    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:22:53.867841 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:53.867810    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:54.029619 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:54.029588    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:54.131104 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:54.131034    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:54.286232 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:54.286206    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:54.537816 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:54.537736    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:54.634489 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:54.634451    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:55.710224 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.710189    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"]
Oct 30 23:22:55.713968 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.713947    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.719722 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.719698    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-gateway-2-openshift-default-dockercfg-747c2\""
Oct 30 23:22:55.719890 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.719764    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Oct 30 23:22:55.719890 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.719786    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Oct 30 23:22:55.719890 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.719889    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-gw-ca-root-cert\""
Oct 30 23:22:55.730118 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.730095    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"]
Oct 30 23:22:55.765198 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.765172    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:55.861832 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.861794    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.861832 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.861836    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-socket\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.862067 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.861903    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92lgj\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-kube-api-access-92lgj\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.862067 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.861980    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-envoy\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.862067 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.862012    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-token\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.862229 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.862114    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-certs\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.862229 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.862155    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-credential-socket\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.862293 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.862249    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-podinfo\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.862328 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.862294    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-data\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.874108 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.874086    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:55.963095 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963044    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-envoy\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963095 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963071    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-token\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963095 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963092    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-certs\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963294 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963111    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-credential-socket\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963294 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963160    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-podinfo\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963294 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963203    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-data\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963294 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963268    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963294 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963291    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-socket\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963550 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963324    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92lgj\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-kube-api-access-92lgj\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963603 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963584    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-certs\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963702 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963654    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-data\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963870 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963829    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-socket\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.963870 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.963847    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-credential-socket\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.964089 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.964053    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istiod-ca-cert\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.965460 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.965442    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-envoy\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.965768 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.965750    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-podinfo\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.976004 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.975977    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-token\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:55.976534 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:55.976510    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92lgj\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-kube-api-access-92lgj\") pod \"router-gateway-2-openshift-default-7bd678d5fc-bdd9c\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:56.028923 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:56.028903    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:56.308156 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:56.308127    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"]
Oct 30 23:22:56.314513 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:22:56.314486    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd23f0c8_ad91_496a_a7d7_5a80c51ee5e4.slice/crio-c57b9a3f30042a891295571fc185baff62e2f6c7c3efc119ee0651368e4bcc18 WatchSource:0}: Error finding container c57b9a3f30042a891295571fc185baff62e2f6c7c3efc119ee0651368e4bcc18: Status 404 returned error can't find the container with id c57b9a3f30042a891295571fc185baff62e2f6c7c3efc119ee0651368e4bcc18
Oct 30 23:22:56.316521 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:56.316495    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:22:56.316590 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:56.316565    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:22:56.316626 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:56.316595    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:22:56.727954 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:56.727912    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:56.832249 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:56.832220    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:56.985473 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:56.985392    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" event={"ID":"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4","Type":"ContainerStarted","Data":"e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb"}
Oct 30 23:22:56.985473 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:56.985447    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" event={"ID":"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4","Type":"ContainerStarted","Data":"c57b9a3f30042a891295571fc185baff62e2f6c7c3efc119ee0651368e4bcc18"}
Oct 30 23:22:57.016220 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:57.016170    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" podStartSLOduration=2.016157429 podStartE2EDuration="2.016157429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:22:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:22:57.013947837 +0000 UTC m=+3819.283029387" watchObservedRunningTime="2026-10-30 23:22:57.016157429 +0000 UTC m=+3819.285238948"
Oct 30 23:22:57.029239 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:57.029217    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:57.034116 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:57.034096    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:57.612691 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:57.604769    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:22:57.990559 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:57.990479    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:58.006643 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:58.002900    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:22:58.762179 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:22:58.762145    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:00.054098 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:00.054062    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:00.155187 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:00.155160    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:00.318743 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:00.314476    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:01.210228 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:01.210196    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:01.310143 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:01.310071    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:06.067029 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:06.067003    5503 generic.go:356] "Generic (PLEG): container finished" podID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerID="e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb" exitCode=0
Oct 30 23:23:06.067377 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:06.067058    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" event={"ID":"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4","Type":"ContainerDied","Data":"e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb"}
Oct 30 23:23:06.067377 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:06.067313    5503 scope.go:122] "RemoveContainer" containerID="e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb"
Oct 30 23:23:06.068283 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:06.068259    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:23:06.068347 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:06.068323    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:23:06.068383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:06.068354    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:23:06.628189 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:06.628151    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:06.793010 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:06.792973    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:07.074197 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:07.074119    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" event={"ID":"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4","Type":"ContainerStarted","Data":"360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9"}
Oct 30 23:23:08.029496 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.029463    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:23:08.029688 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.029652    5503 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 23:23:08.265558 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.265520    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:08.488442 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.488406    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"]
Oct 30 23:23:08.491533 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.491510    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:08.512699 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.499253    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-epp-sa-dockercfg-fc4rw\""
Oct 30 23:23:08.529354 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.528043    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-mnq7j\""
Oct 30 23:23:08.529354 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.528189    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Oct 30 23:23:08.532497 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.532471    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"]
Oct 30 23:23:08.534545 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.534514    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce74e6cd-10f0-4771-85de-b10860796c13-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl\" (UID: \"ce74e6cd-10f0-4771-85de-b10860796c13\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:08.534678 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.534635    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ldmq\" (UniqueName: \"kubernetes.io/projected/ce74e6cd-10f0-4771-85de-b10860796c13-kube-api-access-8ldmq\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl\" (UID: \"ce74e6cd-10f0-4771-85de-b10860796c13\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:08.553276 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.553253    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:08.637861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.636199    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ldmq\" (UniqueName: \"kubernetes.io/projected/ce74e6cd-10f0-4771-85de-b10860796c13-kube-api-access-8ldmq\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl\" (UID: \"ce74e6cd-10f0-4771-85de-b10860796c13\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:08.637861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.636321    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce74e6cd-10f0-4771-85de-b10860796c13-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl\" (UID: \"ce74e6cd-10f0-4771-85de-b10860796c13\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:08.650955 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.646868    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce74e6cd-10f0-4771-85de-b10860796c13-tls-certs\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl\" (UID: \"ce74e6cd-10f0-4771-85de-b10860796c13\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:08.656695 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.655813    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ldmq\" (UniqueName: \"kubernetes.io/projected/ce74e6cd-10f0-4771-85de-b10860796c13-kube-api-access-8ldmq\") pod \"router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl\" (UID: \"ce74e6cd-10f0-4771-85de-b10860796c13\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:08.663252 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.663226    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:08.845933 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:08.844525    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:09.030181 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:09.030138    5503 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 23:23:09.165503 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:09.165476    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"]
Oct 30 23:23:09.167584 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:23:09.167543    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce74e6cd_10f0_4771_85de_b10860796c13.slice/crio-0d289464a3a3425b191c448a848984ccdb2d1e1c64daacf0599f3012b28739e6 WatchSource:0}: Error finding container 0d289464a3a3425b191c448a848984ccdb2d1e1c64daacf0599f3012b28739e6: Status 404 returned error can't find the container with id 0d289464a3a3425b191c448a848984ccdb2d1e1c64daacf0599f3012b28739e6
Oct 30 23:23:10.029907 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:10.029861    5503 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 23:23:10.092659 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:10.092619    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" event={"ID":"ce74e6cd-10f0-4771-85de-b10860796c13","Type":"ContainerStarted","Data":"f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097"}
Oct 30 23:23:10.092842 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:10.092711    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" event={"ID":"ce74e6cd-10f0-4771-85de-b10860796c13","Type":"ContainerStarted","Data":"0d289464a3a3425b191c448a848984ccdb2d1e1c64daacf0599f3012b28739e6"}
Oct 30 23:23:10.092842 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:10.092745    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:10.122368 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:10.122314    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" podStartSLOduration=2.12230158 podStartE2EDuration="2.12230158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:23:10.1210043 +0000 UTC m=+3832.390085822" watchObservedRunningTime="2026-10-30 23:23:10.12230158 +0000 UTC m=+3832.391383101"
Oct 30 23:23:11.030206 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:11.030164    5503 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused"
Oct 30 23:23:12.033252 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:12.033216    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:23:12.033740 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:12.033501    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:23:12.034150 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:12.034134    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:23:15.388644 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:23:15.388608    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:23:16.891262 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:16.891225    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:23.122270 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:23.122231    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:23:41.110166 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:23:41.110137    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:23:45.388406 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:23:45.388372    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:24:15.390230 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:24:15.390188    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:24:16.953010 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:24:16.952975    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:24:45.389547 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:24:45.389509    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:24:57.249988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:24:57.249936    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:15.388906 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:25:15.388868    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:25:17.024807 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:17.024778    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:39.050619 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:39.050542    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:40.422641 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:40.422614    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:40.443420 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:40.443387    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"]
Oct 30 23:25:40.527029 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:40.526999    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:41.110146 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:41.110116    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:25:45.388478 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:25:45.388443    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:25:51.109952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:51.109922    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:25:55.464548 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:55.464514    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" containerName="main" containerID="cri-o://f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097" gracePeriod=45
Oct 30 23:25:57.300325 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:57.300288    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:59.220701 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:59.220647    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:59.404187 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:59.404151    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:59.513753 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:59.513688    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:59.835182 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:59.835152    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:25:59.963806 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:25:59.963776    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:01.110054 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:01.110014    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:26:01.110489 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:01.110125    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:26:02.568994 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:02.568965    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:03.741537 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:03.741510    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:05.797191 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:05.797159    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:05.908239 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:05.908215    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:08.951685 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:08.951636    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:09.055582 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:09.055558    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:11.109961 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:11.109921    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:26:11.680880 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:11.680854    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:11.783309 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:11.783285    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:15.388812 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:26:15.388776    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:26:17.116850 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:17.116813    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:17.251806 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:17.251777    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:21.109470 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:21.109436    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:26:25.713087 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:25.713062    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:26:25.795090 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:25.795036    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce74e6cd-10f0-4771-85de-b10860796c13-kube-api-access-8ldmq\" (UniqueName: \"kubernetes.io/projected/ce74e6cd-10f0-4771-85de-b10860796c13-kube-api-access-8ldmq\") pod \"ce74e6cd-10f0-4771-85de-b10860796c13\" (UID: \"ce74e6cd-10f0-4771-85de-b10860796c13\") "
Oct 30 23:26:25.795198 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:25.795122    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce74e6cd-10f0-4771-85de-b10860796c13-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce74e6cd-10f0-4771-85de-b10860796c13-tls-certs\") pod \"ce74e6cd-10f0-4771-85de-b10860796c13\" (UID: \"ce74e6cd-10f0-4771-85de-b10860796c13\") "
Oct 30 23:26:25.797373 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:25.797343    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce74e6cd-10f0-4771-85de-b10860796c13-tls-certs" pod "ce74e6cd-10f0-4771-85de-b10860796c13" (UID: "ce74e6cd-10f0-4771-85de-b10860796c13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:26:25.797467 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:25.797347    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce74e6cd-10f0-4771-85de-b10860796c13-kube-api-access-8ldmq" pod "ce74e6cd-10f0-4771-85de-b10860796c13" (UID: "ce74e6cd-10f0-4771-85de-b10860796c13"). InnerVolumeSpecName "kube-api-access-8ldmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:26:25.895995 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:25.895969    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce74e6cd-10f0-4771-85de-b10860796c13-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:25.895995 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:25.895993    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ldmq\" (UniqueName: \"kubernetes.io/projected/ce74e6cd-10f0-4771-85de-b10860796c13-kube-api-access-8ldmq\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:26.373727 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:26.373697    5503 generic.go:356] "Generic (PLEG): container finished" podID="ce74e6cd-10f0-4771-85de-b10860796c13" containerID="f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097" exitCode=0
Oct 30 23:26:26.373861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:26.373739    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"
Oct 30 23:26:26.373861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:26.373779    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" event={"ID":"ce74e6cd-10f0-4771-85de-b10860796c13","Type":"ContainerDied","Data":"f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097"}
Oct 30 23:26:26.373861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:26.373830    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl" event={"ID":"ce74e6cd-10f0-4771-85de-b10860796c13","Type":"ContainerDied","Data":"0d289464a3a3425b191c448a848984ccdb2d1e1c64daacf0599f3012b28739e6"}
Oct 30 23:26:26.373861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:26.373851    5503 scope.go:122] "RemoveContainer" containerID="f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097"
Oct 30 23:26:26.382573 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:26.382557    5503 scope.go:122] "RemoveContainer" containerID="f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097"
Oct 30 23:26:26.382857 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:26:26.382833    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097\": container with ID starting with f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097 not found: ID does not exist" containerID="f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097"
Oct 30 23:26:26.382919 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:26.382869    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097"} err="failed to get container status \"f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097\": rpc error: code = NotFound desc = could not find container \"f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097\": container with ID starting with f4aeece1e0eb62d6264d6cace37190ae1a9071cface502b256e5b8cf0b893097 not found: ID does not exist"
Oct 30 23:26:26.392705 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:26.392684    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"]
Oct 30 23:26:26.397219 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:26.397200    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-router-scheduler-6988f9c7d2zpbl"]
Oct 30 23:26:27.855575 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:27.855545    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" path="/var/lib/kubelet/pods/ce74e6cd-10f0-4771-85de-b10860796c13/volumes"
Oct 30 23:26:30.201507 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:30.201472    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:35.155146 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:35.155107    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"]
Oct 30 23:26:35.155610 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:35.155353    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy" containerID="cri-o://360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9" gracePeriod=30
Oct 30 23:26:35.228597 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:35.228560    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:40.403790 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.403762    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:26:40.441568 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.441492    5503 generic.go:356] "Generic (PLEG): container finished" podID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerID="360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9" exitCode=0
Oct 30 23:26:40.441568 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.441558    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"
Oct 30 23:26:40.441793 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.441577    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" event={"ID":"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4","Type":"ContainerDied","Data":"360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9"}
Oct 30 23:26:40.441793 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.441631    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c" event={"ID":"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4","Type":"ContainerDied","Data":"c57b9a3f30042a891295571fc185baff62e2f6c7c3efc119ee0651368e4bcc18"}
Oct 30 23:26:40.441793 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.441654    5503 scope.go:122] "RemoveContainer" containerID="360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9"
Oct 30 23:26:40.450878 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.450861    5503 scope.go:122] "RemoveContainer" containerID="e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb"
Oct 30 23:26:40.461325 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.461300    5503 scope.go:122] "RemoveContainer" containerID="360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9"
Oct 30 23:26:40.461634 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:26:40.461608    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9\": container with ID starting with 360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9 not found: ID does not exist" containerID="360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9"
Oct 30 23:26:40.461729 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.461649    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9"} err="failed to get container status \"360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9\": rpc error: code = NotFound desc = could not find container \"360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9\": container with ID starting with 360aec1eca7b15e81ab2f28feda0d2b851df36ef1f415f5f37ef2792371897d9 not found: ID does not exist"
Oct 30 23:26:40.461729 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.461700    5503 scope.go:122] "RemoveContainer" containerID="e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb"
Oct 30 23:26:40.462052 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:26:40.462022    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb\": container with ID starting with e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb not found: ID does not exist" containerID="e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb"
Oct 30 23:26:40.462146 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.462060    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb"} err="failed to get container status \"e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb\": rpc error: code = NotFound desc = could not find container \"e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb\": container with ID starting with e0c52d14da8e50aed94bd8ae515c5a5212fc3ea6866b3dbc4044de2b40a19abb not found: ID does not exist"
Oct 30 23:26:40.515540 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515517    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istiod-ca-cert\") pod \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") "
Oct 30 23:26:40.515651 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515564    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-envoy\") pod \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") "
Oct 30 23:26:40.515651 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515587    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-podinfo\") pod \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") "
Oct 30 23:26:40.515651 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515604    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-credential-socket\") pod \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") "
Oct 30 23:26:40.515843 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515740    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-data\") pod \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") "
Oct 30 23:26:40.515843 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515837    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-token\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-token\") pod \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") "
Oct 30 23:26:40.515954 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515903    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-credential-socket" pod "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" (UID: "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:26:40.515954 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515906    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-kube-api-access-92lgj\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-kube-api-access-92lgj\") pod \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") "
Oct 30 23:26:40.516058 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515979    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-data" pod "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" (UID: "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:26:40.516058 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515989    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-socket\") pod \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") "
Oct 30 23:26:40.516058 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.515990    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istiod-ca-cert" pod "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" (UID: "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:26:40.516058 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.516035    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-certs\") pod \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\" (UID: \"dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4\") "
Oct 30 23:26:40.516294 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.516253    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-socket" pod "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" (UID: "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:26:40.516409 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.516390    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:40.516409 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.516410    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:40.516553 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.516425    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:40.516553 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.516436    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:40.516553 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.516409    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-certs" pod "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" (UID: "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:26:40.517862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.517840    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-podinfo" pod "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" (UID: "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:26:40.518118 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.518104    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-kube-api-access-92lgj" pod "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" (UID: "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4"). InnerVolumeSpecName "kube-api-access-92lgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:26:40.518305 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.518283    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-token" pod "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" (UID: "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:26:40.518385 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.518301    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-envoy" pod "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" (UID: "dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:26:40.617555 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.617528    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:40.617555 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.617552    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:40.617736 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.617562    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92lgj\" (UniqueName: \"kubernetes.io/projected/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-kube-api-access-92lgj\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:40.617736 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.617572    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:40.617736 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.617582    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:26:40.766207 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.766180    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"]
Oct 30 23:26:40.770634 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:40.770613    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-gateway-2-openshift-default-7bd678d5fc-bdd9c"]
Oct 30 23:26:41.854569 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:41.854536    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" path="/var/lib/kubelet/pods/dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4/volumes"
Oct 30 23:26:45.388427 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:26:45.388393    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:26:50.295832 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:50.295799    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:56.579655 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:56.579608    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:26:57.989837 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:26:57.989810    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:03.883954 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:03.883920    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:07.852783 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:07.852748    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:09.840529 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:09.840503    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:09.960864 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:09.960833    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:15.387886 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:27:15.387850    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:27:16.182493 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.182458    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:16.334365 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.334334    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:16.466639 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.466570    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:16.644692 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.644648    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"]
Oct 30 23:27:16.645115 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.645095    5503 cpu_manager.go:414] "containerMap: removing container" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy"
Oct 30 23:27:16.645115 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.645113    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy"
Oct 30 23:27:16.645266 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.645129    5503 cpu_manager.go:414] "containerMap: removing container" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy"
Oct 30 23:27:16.645266 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.645135    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy"
Oct 30 23:27:16.645266 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.645166    5503 cpu_manager.go:414] "containerMap: removing container" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" containerName="main"
Oct 30 23:27:16.645266 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.645172    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" containerName="main"
Oct 30 23:27:16.645266 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.645239    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce74e6cd-10f0-4771-85de-b10860796c13" containerName="main"
Oct 30 23:27:16.645266 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.645250    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy"
Oct 30 23:27:16.645266 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.645257    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="dd23f0c8-ad91-496a-a7d7-5a80c51ee5e4" containerName="istio-proxy"
Oct 30 23:27:16.648305 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.648286    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.652893 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.652867    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"openshift-service-ca.crt\""
Oct 30 23:27:16.652893 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.652868    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"openshift-gw-ca-root-cert\""
Oct 30 23:27:16.653098 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.653028    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"model-routing-only-gateway-openshift-default-dockercfg-rxlcd\""
Oct 30 23:27:16.666862 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.666837    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"]
Oct 30 23:27:16.703273 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.703239    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-certs\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.703273 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.703272    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-data\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.703453 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.703297    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-socket\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.703453 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.703373    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-envoy\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.703569 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.703456    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-podinfo\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.703569 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.703535    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-token\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.703688 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.703583    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-credential-socket\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.703688 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.703627    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istiod-ca-cert\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.703799 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.703724    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmt4j\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-kube-api-access-hmt4j\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.755960 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.755899    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-fae765e6\"/\"kube-root-ca.crt\""
Oct 30 23:27:16.787588 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.787548    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:16.805027 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.804997    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-podinfo\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805175 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805031    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-token\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805175 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805059    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-credential-socket\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805175 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805088    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istiod-ca-cert\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805175 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805119    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmt4j\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-kube-api-access-hmt4j\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805395 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805178    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-certs\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805395 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805289    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-data\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805395 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805377    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-socket\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805519 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805406    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-envoy\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805557 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805512    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-credential-socket\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805557 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805541    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-certs\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805655 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805619    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-data\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.805747 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.805731    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-socket\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.806037 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.806015    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istiod-ca-cert\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.807840 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.807820    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-podinfo\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.807954 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.807834    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-envoy\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.822996 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.822975    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmt4j\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-kube-api-access-hmt4j\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.832233 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.832204    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-token\") pod \"model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") " pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:16.952225 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.952195    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:16.964216 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:16.964029    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:17.255090 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:17.255055    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:17.266141 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:17.266118    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"]
Oct 30 23:27:17.267456 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:27:17.267431    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d73c9a_f930_4c5b_9f5b_d24ed9bddea6.slice/crio-f6197d7830a5a86b693d1a2028b1d47a29d617c32f11fc282c261b2f9a9794b6 WatchSource:0}: Error finding container f6197d7830a5a86b693d1a2028b1d47a29d617c32f11fc282c261b2f9a9794b6: Status 404 returned error can't find the container with id f6197d7830a5a86b693d1a2028b1d47a29d617c32f11fc282c261b2f9a9794b6
Oct 30 23:27:17.269362 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:17.269336    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:27:17.269421 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:17.269393    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:27:17.269456 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:17.269421    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:27:17.624399 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:17.624367    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" event={"ID":"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6","Type":"ContainerStarted","Data":"f8c900cf5ceadcf6251febcc5e9eb73d2911bbbbfbaebfbeef2a96cfe878fef9"}
Oct 30 23:27:17.624794 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:17.624428    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" event={"ID":"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6","Type":"ContainerStarted","Data":"f6197d7830a5a86b693d1a2028b1d47a29d617c32f11fc282c261b2f9a9794b6"}
Oct 30 23:27:17.652803 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:17.652753    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" podStartSLOduration=1.6527342969999999 podStartE2EDuration="1.652734297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:27:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:27:17.64950339 +0000 UTC m=+4079.918584931" watchObservedRunningTime="2026-10-30 23:27:17.652734297 +0000 UTC m=+4079.921815821"
Oct 30 23:27:17.964504 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:17.964427    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:18.969382 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:18.969357    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:19.418089 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:19.418061    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:19.634852 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:19.634775    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:19.635653 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:19.635630    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:20.053422 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:20.053338    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:21.242630 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:21.242594    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:27.960958 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:27.960923    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:28.541230 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:28.541198    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:29.683971 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:29.683917    5503 generic.go:356] "Generic (PLEG): container finished" podID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerID="f8c900cf5ceadcf6251febcc5e9eb73d2911bbbbfbaebfbeef2a96cfe878fef9" exitCode=0
Oct 30 23:27:29.683971 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:29.683958    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" event={"ID":"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6","Type":"ContainerDied","Data":"f8c900cf5ceadcf6251febcc5e9eb73d2911bbbbfbaebfbeef2a96cfe878fef9"}
Oct 30 23:27:29.684330 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:29.684243    5503 scope.go:122] "RemoveContainer" containerID="f8c900cf5ceadcf6251febcc5e9eb73d2911bbbbfbaebfbeef2a96cfe878fef9"
Oct 30 23:27:29.685228 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:29.685199    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:27:29.685286 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:29.685273    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:27:29.685320 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:29.685308    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:27:30.690619 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:30.690584    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" event={"ID":"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6","Type":"ContainerStarted","Data":"08ca2428be50054d6c0dfa00b8b20a45f74c65770d6f003d787796b6a7db1c11"}
Oct 30 23:27:30.964531 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:30.964453    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:30.965624 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:30.965595    5503 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 23:27:31.964987 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:31.964949    5503 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 23:27:32.964736 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:32.964691    5503 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 23:27:33.964951 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:33.964915    5503 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 23:27:34.635744 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:34.635712    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:34.965173 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:34.965090    5503 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused"
Oct 30 23:27:35.967871 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:35.967846    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:35.968812 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:35.968792    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:27:42.926374 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:42.926337    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:43.037523 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:43.037481    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:44.187161 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:44.187129    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:45.390765 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:27:45.390717    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:27:50.352175 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:50.352142    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:58.393936 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:58.393902    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:27:59.102535 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:27:59.102503    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:02.370726 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:02.370685    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:05.319481 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:05.319445    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:05.420231 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:05.420199    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:15.388499 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:28:15.388466    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:28:17.214225 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:17.214188    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:18.830640 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:18.830610    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:32.952472 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:32.952389    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:33.056444 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:33.056411    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:37.914788 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:37.914755    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"]
Oct 30 23:28:37.915257 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:37.914915    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy" containerID="cri-o://08ca2428be50054d6c0dfa00b8b20a45f74c65770d6f003d787796b6a7db1c11" gracePeriod=30
Oct 30 23:28:38.013632 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:38.013603    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:39.750299 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:39.750265    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:39.873583 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:39.873550    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:43.055483 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.055454    5503 generic.go:356] "Generic (PLEG): container finished" podID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerID="08ca2428be50054d6c0dfa00b8b20a45f74c65770d6f003d787796b6a7db1c11" exitCode=0
Oct 30 23:28:43.055838 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.055495    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" event={"ID":"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6","Type":"ContainerDied","Data":"08ca2428be50054d6c0dfa00b8b20a45f74c65770d6f003d787796b6a7db1c11"}
Oct 30 23:28:43.055838 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.055543    5503 scope.go:122] "RemoveContainer" containerID="f8c900cf5ceadcf6251febcc5e9eb73d2911bbbbfbaebfbeef2a96cfe878fef9"
Oct 30 23:28:43.164742 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.164719    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:28:43.232460 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.232424    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-credential-socket\") pod \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") "
Oct 30 23:28:43.232725 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.232688    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-kube-api-access-hmt4j\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-kube-api-access-hmt4j\") pod \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") "
Oct 30 23:28:43.232887 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.232854    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-token\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-token\") pod \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") "
Oct 30 23:28:43.233023 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.232901    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-socket\") pod \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") "
Oct 30 23:28:43.233023 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.233011    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-certs\") pod \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") "
Oct 30 23:28:43.233142 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.233049    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-podinfo\") pod \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") "
Oct 30 23:28:43.233142 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.233105    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istiod-ca-cert\") pod \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") "
Oct 30 23:28:43.233250 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.233170    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-envoy\") pod \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") "
Oct 30 23:28:43.233250 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.233209    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-data\") pod \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\" (UID: \"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6\") "
Oct 30 23:28:43.233891 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.232729    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-credential-socket" pod "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" (UID: "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:28:43.233891 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.233526    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-socket" pod "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" (UID: "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:28:43.233891 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.233792    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-data" pod "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" (UID: "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:28:43.234115 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.234049    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-certs" pod "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" (UID: "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:28:43.234319 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.234293    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istiod-ca-cert" pod "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" (UID: "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:28:43.235718 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.235683    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-kube-api-access-hmt4j" pod "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" (UID: "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6"). InnerVolumeSpecName "kube-api-access-hmt4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:28:43.236613 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.236587    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-envoy" pod "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" (UID: "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:28:43.236863 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.236838    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-podinfo" pod "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" (UID: "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:28:43.236962 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.236934    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-token" pod "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" (UID: "54d73c9a-f930-4c5b-9f5b-d24ed9bddea6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:28:43.334212 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.334179    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:28:43.334212 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.334207    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:28:43.334391 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.334220    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:28:43.334391 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.334232    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:28:43.334391 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.334243    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:28:43.334391 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.334254    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:28:43.334391 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.334266    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmt4j\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-kube-api-access-hmt4j\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:28:43.334391 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.334278    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:28:43.334391 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.334290    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:28:43.725524 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:43.725487    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:44.061166 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:44.061100    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs" event={"ID":"54d73c9a-f930-4c5b-9f5b-d24ed9bddea6","Type":"ContainerDied","Data":"f6197d7830a5a86b693d1a2028b1d47a29d617c32f11fc282c261b2f9a9794b6"}
Oct 30 23:28:44.061166 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:44.061129    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"
Oct 30 23:28:44.061166 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:44.061147    5503 scope.go:122] "RemoveContainer" containerID="08ca2428be50054d6c0dfa00b8b20a45f74c65770d6f003d787796b6a7db1c11"
Oct 30 23:28:44.083554 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:44.083522    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"]
Oct 30 23:28:44.090509 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:44.090487    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-model-based-routing-only-fae765e6/model-routing-only-gateway-openshift-default-5bd7d9db66-bwrxs"]
Oct 30 23:28:45.482703 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:28:45.475136    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:28:45.870691 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:45.869809    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" path="/var/lib/kubelet/pods/54d73c9a-f930-4c5b-9f5b-d24ed9bddea6/volumes"
Oct 30 23:28:49.147689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.107185    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:49.347689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.346658    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"]
Oct 30 23:28:49.347689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.347265    5503 cpu_manager.go:414] "containerMap: removing container" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy"
Oct 30 23:28:49.347689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.347279    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy"
Oct 30 23:28:49.347689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.347296    5503 cpu_manager.go:414] "containerMap: removing container" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy"
Oct 30 23:28:49.347689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.347304    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy"
Oct 30 23:28:49.347689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.347388    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy"
Oct 30 23:28:49.347689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.347398    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="54d73c9a-f930-4c5b-9f5b-d24ed9bddea6" containerName="istio-proxy"
Oct 30 23:28:49.395321 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.394044    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:28:49.411988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.411909    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"tls-verification-test-epp-sa-dockercfg-67cr4\""
Oct 30 23:28:49.421692 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.412251    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"tls-verification-test-kserve-self-signed-certs\""
Oct 30 23:28:49.427028 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.414837    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"openshift-service-ca.crt\""
Oct 30 23:28:49.431746 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.414879    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"default-dockercfg-fjgbt\""
Oct 30 23:28:49.431913 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.414960    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"kube-root-ca.crt\""
Oct 30 23:28:49.432048 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.432033    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"]
Oct 30 23:28:49.481693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.470796    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1ad02-e855-4430-abd6-9435d5a0afa7-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5\" (UID: \"86b1ad02-e855-4430-abd6-9435d5a0afa7\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:28:49.481693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.470876    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rd9h\" (UniqueName: \"kubernetes.io/projected/86b1ad02-e855-4430-abd6-9435d5a0afa7-kube-api-access-4rd9h\") pod \"tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5\" (UID: \"86b1ad02-e855-4430-abd6-9435d5a0afa7\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:28:49.481693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.473166    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:49.574069 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.573198    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1ad02-e855-4430-abd6-9435d5a0afa7-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5\" (UID: \"86b1ad02-e855-4430-abd6-9435d5a0afa7\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:28:49.574069 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.573268    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rd9h\" (UniqueName: \"kubernetes.io/projected/86b1ad02-e855-4430-abd6-9435d5a0afa7-kube-api-access-4rd9h\") pod \"tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5\" (UID: \"86b1ad02-e855-4430-abd6-9435d5a0afa7\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:28:49.610689 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.610093    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rd9h\" (UniqueName: \"kubernetes.io/projected/86b1ad02-e855-4430-abd6-9435d5a0afa7-kube-api-access-4rd9h\") pod \"tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5\" (UID: \"86b1ad02-e855-4430-abd6-9435d5a0afa7\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:28:49.638691 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.635539    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1ad02-e855-4430-abd6-9435d5a0afa7-tls-certs\") pod \"tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5\" (UID: \"86b1ad02-e855-4430-abd6-9435d5a0afa7\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:28:49.868246 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:49.867944    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:28:50.792574 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:50.792479    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"]
Oct 30 23:28:51.303639 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:51.302688    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" event={"ID":"86b1ad02-e855-4430-abd6-9435d5a0afa7","Type":"ContainerStarted","Data":"1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1"}
Oct 30 23:28:51.303639 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:51.302758    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" event={"ID":"86b1ad02-e855-4430-abd6-9435d5a0afa7","Type":"ContainerStarted","Data":"7a9deb8ac636d24d98db472460ba9dcc8b4ece21bdb9a1f57b5f8de6d8927877"}
Oct 30 23:28:51.303639 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:51.303561    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:28:58.125006 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:58.124975    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:59.141475 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:59.141433    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:28:59.243017 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:28:59.242988    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:01.219988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:01.219956    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:02.404746 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:02.404708    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:07.333272 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:07.333239    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:15.388193 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:29:15.388157    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:29:17.491174 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:17.491142    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:20.367498 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:20.367466    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:22.090856 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:22.090826    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:22.191612 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:22.191585    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:23.315088 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:23.315063    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:29:23.341614 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:23.341570    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" podStartSLOduration=34.341557215 podStartE2EDuration="34.341557215s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:28:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:28:51.336817084 +0000 UTC m=+4173.605898609" watchObservedRunningTime="2026-10-30 23:29:23.341557215 +0000 UTC m=+4205.610638736"
Oct 30 23:29:27.307944 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:27.307908    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:39.806245 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:39.806215    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:45.388609 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:29:45.388577    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:29:46.613499 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:46.613469    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:49.785094 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:49.785060    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:29:49.882445 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:29:49.882420    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:30:02.480736 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:30:02.480703    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:30:15.388925 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:30:15.388890    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:30:27.408630 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:30:27.408594    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:30:45.389418 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:30:45.389377    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:31:02.524984 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:31:02.524925    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:31:15.388348 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:31:15.388300    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:31:22.384936 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:31:22.384901    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:31:27.429077 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:31:27.428993    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:31:40.795978 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:31:40.795919    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:31:45.389345 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:31:45.389309    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:32:00.981270 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:32:00.981238    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:32:01.085822 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:32:01.085798    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:32:07.116688 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:32:07.116633    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:32:12.569937 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:32:12.569900    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:32:12.994030 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:32:12.993996    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:32:13.099646 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:32:13.099618    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:32:15.389596 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:32:15.389561    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:32:22.419853 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:32:22.419817    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:32:37.532269 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:32:37.532232    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:32:45.388621 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:32:45.388583    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:33:12.637808 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:33:12.637770    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:33:15.389295 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:33:15.389247    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:33:22.477778 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:33:22.477745    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:33:37.543864 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:33:37.543830    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:33:44.718884 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:33:44.718845    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:33:45.388906 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:33:45.388872    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:33:47.394358 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:33:47.394313    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-tls-resources-e4cad0c3_tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5_86b1ad02-e855-4430-abd6-9435d5a0afa7/main/0.log"
Oct 30 23:33:51.129887 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:33:51.129844    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:33:51.144464 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:33:51.144430    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"]
Oct 30 23:33:51.243839 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:33:51.243804    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:33:53.315455 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:33:53.315420    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:34:03.316082 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:03.316032    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:34:06.171898 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:06.171845    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerName="main" containerID="cri-o://1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1" gracePeriod=45
Oct 30 23:34:08.823759 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:08.823725    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:11.460410 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:11.460372    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"]
Oct 30 23:34:11.460829 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:11.460575    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy" containerID="cri-o://8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4" gracePeriod=30
Oct 30 23:34:11.466128 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:11.466101    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"]
Oct 30 23:34:11.467736 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:11.467708    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"]
Oct 30 23:34:11.467992 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:11.467964    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy" containerID="cri-o://24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd" gracePeriod=30
Oct 30 23:34:11.469630 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:11.469602    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"]
Oct 30 23:34:11.470014 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:11.469946    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy" containerID="cri-o://ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d" gracePeriod=30
Oct 30 23:34:11.840063 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:11.839981    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy" containerID="cri-o://43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d" gracePeriod=30
Oct 30 23:34:12.678955 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:12.678925    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:13.315364 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:13.315327    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:34:13.315513 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:13.315421    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:34:15.388682 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:34:15.388632    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:34:16.487557 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.487523    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-c7w8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.72:15021/healthz/ready\": read tcp 10.133.0.2:52642->10.133.0.72:15021: read: connection reset by peer" start-of-body=
Oct 30 23:34:16.487913 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.487578    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.72:15021/healthz/ready\": read tcp 10.133.0.2:52642->10.133.0.72:15021: read: connection reset by peer"
Oct 30 23:34:16.501650 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.501626    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-fbb5t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused" start-of-body=
Oct 30 23:34:16.501774 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.501685    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.73:15021/healthz/ready\": dial tcp 10.133.0.73:15021: connect: connection refused"
Oct 30 23:34:16.522378 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.522353    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-xfb4p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Oct 30 23:34:16.522460 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.522390    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Oct 30 23:34:16.721980 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.721953    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:34:16.765993 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.765972    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:34:16.769606 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.769582    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:34:16.810486 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810458    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-token\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-token\") pod \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") "
Oct 30 23:34:16.810597 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810494    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-data\") pod \"f059a50f-09a7-46bf-8ba8-de3332be620f\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") "
Oct 30 23:34:16.810597 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810525    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-socket\") pod \"f059a50f-09a7-46bf-8ba8-de3332be620f\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") "
Oct 30 23:34:16.810597 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810545    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-credential-socket\") pod \"2afbb22b-8fa2-4695-b392-3a99853dfd94\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") "
Oct 30 23:34:16.810776 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810629    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-credential-socket\") pod \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") "
Oct 30 23:34:16.810776 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810702    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-certs\") pod \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") "
Oct 30 23:34:16.810776 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810734    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-data\") pod \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") "
Oct 30 23:34:16.810776 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810770    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-envoy\") pod \"f059a50f-09a7-46bf-8ba8-de3332be620f\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") "
Oct 30 23:34:16.810972 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810803    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-certs\") pod \"2afbb22b-8fa2-4695-b392-3a99853dfd94\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") "
Oct 30 23:34:16.810972 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810833    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-credential-socket\") pod \"f059a50f-09a7-46bf-8ba8-de3332be620f\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") "
Oct 30 23:34:16.810972 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810873    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-kube-api-access-m7lzm\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-kube-api-access-m7lzm\") pod \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") "
Oct 30 23:34:16.810972 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810901    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-token\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-token\") pod \"f059a50f-09a7-46bf-8ba8-de3332be620f\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") "
Oct 30 23:34:16.810972 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810951    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-certs\") pod \"f059a50f-09a7-46bf-8ba8-de3332be620f\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") "
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811009    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-socket\") pod \"2afbb22b-8fa2-4695-b392-3a99853dfd94\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") "
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811046    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2afbb22b-8fa2-4695-b392-3a99853dfd94-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2afbb22b-8fa2-4695-b392-3a99853dfd94-istiod-ca-cert\") pod \"2afbb22b-8fa2-4695-b392-3a99853dfd94\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") "
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811097    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-envoy\") pod \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") "
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811123    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-socket\") pod \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") "
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810832    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-socket" pod "f059a50f-09a7-46bf-8ba8-de3332be620f" (UID: "f059a50f-09a7-46bf-8ba8-de3332be620f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810831    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-credential-socket" pod "2afbb22b-8fa2-4695-b392-3a99853dfd94" (UID: "2afbb22b-8fa2-4695-b392-3a99853dfd94"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811160    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-kube-api-access-9d2x7\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-kube-api-access-9d2x7\") pod \"2afbb22b-8fa2-4695-b392-3a99853dfd94\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") "
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810850    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-data" pod "f059a50f-09a7-46bf-8ba8-de3332be620f" (UID: "f059a50f-09a7-46bf-8ba8-de3332be620f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.810976    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-certs" pod "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" (UID: "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811004    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-credential-socket" pod "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" (UID: "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811114    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-certs" pod "2afbb22b-8fa2-4695-b392-3a99853dfd94" (UID: "2afbb22b-8fa2-4695-b392-3a99853dfd94"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.811222 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811131    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-credential-socket" pod "f059a50f-09a7-46bf-8ba8-de3332be620f" (UID: "f059a50f-09a7-46bf-8ba8-de3332be620f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811189    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-envoy\") pod \"2afbb22b-8fa2-4695-b392-3a99853dfd94\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") "
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811333    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istiod-ca-cert\") pod \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") "
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811392    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-data" pod "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" (UID: "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811399    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-certs" pod "f059a50f-09a7-46bf-8ba8-de3332be620f" (UID: "f059a50f-09a7-46bf-8ba8-de3332be620f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811406    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f059a50f-09a7-46bf-8ba8-de3332be620f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f059a50f-09a7-46bf-8ba8-de3332be620f-istiod-ca-cert\") pod \"f059a50f-09a7-46bf-8ba8-de3332be620f\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") "
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811466    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-data\") pod \"2afbb22b-8fa2-4695-b392-3a99853dfd94\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") "
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811494    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-podinfo\") pod \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\" (UID: \"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f\") "
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811573    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-socket" pod "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" (UID: "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811594    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-kube-api-access-ls9g8\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-kube-api-access-ls9g8\") pod \"f059a50f-09a7-46bf-8ba8-de3332be620f\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") "
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811658    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-podinfo\") pod \"2afbb22b-8fa2-4695-b392-3a99853dfd94\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") "
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811722    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-token\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-token\") pod \"2afbb22b-8fa2-4695-b392-3a99853dfd94\" (UID: \"2afbb22b-8fa2-4695-b392-3a99853dfd94\") "
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811770    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-podinfo\") pod \"f059a50f-09a7-46bf-8ba8-de3332be620f\" (UID: \"f059a50f-09a7-46bf-8ba8-de3332be620f\") "
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811608    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-socket" pod "2afbb22b-8fa2-4695-b392-3a99853dfd94" (UID: "2afbb22b-8fa2-4695-b392-3a99853dfd94"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811741    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f059a50f-09a7-46bf-8ba8-de3332be620f-istiod-ca-cert" pod "f059a50f-09a7-46bf-8ba8-de3332be620f" (UID: "f059a50f-09a7-46bf-8ba8-de3332be620f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.811914    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2afbb22b-8fa2-4695-b392-3a99853dfd94-istiod-ca-cert" pod "2afbb22b-8fa2-4695-b392-3a99853dfd94" (UID: "2afbb22b-8fa2-4695-b392-3a99853dfd94"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812015    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istiod-ca-cert" pod "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" (UID: "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812178    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812201    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812219    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812233    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812246    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812259    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812271    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812284    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812296    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812307    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812320    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2afbb22b-8fa2-4695-b392-3a99853dfd94-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812332    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812346    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.812436 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812360    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f059a50f-09a7-46bf-8ba8-de3332be620f-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.814231 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.812948    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-data" pod "2afbb22b-8fa2-4695-b392-3a99853dfd94" (UID: "2afbb22b-8fa2-4695-b392-3a99853dfd94"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.814231 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.813486    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-token" pod "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" (UID: "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:16.814571 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.814544    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-token" pod "f059a50f-09a7-46bf-8ba8-de3332be620f" (UID: "f059a50f-09a7-46bf-8ba8-de3332be620f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:16.815197 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.815150    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-envoy" pod "f059a50f-09a7-46bf-8ba8-de3332be620f" (UID: "f059a50f-09a7-46bf-8ba8-de3332be620f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.815452 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.815420    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-kube-api-access-m7lzm" pod "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" (UID: "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f"). InnerVolumeSpecName "kube-api-access-m7lzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:16.815949 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.815930    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-podinfo" pod "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" (UID: "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:34:16.816884 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.816869    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-kube-api-access-9d2x7" pod "2afbb22b-8fa2-4695-b392-3a99853dfd94" (UID: "2afbb22b-8fa2-4695-b392-3a99853dfd94"). InnerVolumeSpecName "kube-api-access-9d2x7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:16.817599 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.817574    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-kube-api-access-ls9g8" pod "f059a50f-09a7-46bf-8ba8-de3332be620f" (UID: "f059a50f-09a7-46bf-8ba8-de3332be620f"). InnerVolumeSpecName "kube-api-access-ls9g8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:16.817710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.817573    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-envoy" pod "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" (UID: "e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.817710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.817581    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-podinfo" pod "2afbb22b-8fa2-4695-b392-3a99853dfd94" (UID: "2afbb22b-8fa2-4695-b392-3a99853dfd94"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:34:16.817815 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.817758    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-envoy" pod "2afbb22b-8fa2-4695-b392-3a99853dfd94" (UID: "2afbb22b-8fa2-4695-b392-3a99853dfd94"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.817884 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.817868    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-podinfo" pod "f059a50f-09a7-46bf-8ba8-de3332be620f" (UID: "f059a50f-09a7-46bf-8ba8-de3332be620f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:34:16.818874 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.818857    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-token" pod "2afbb22b-8fa2-4695-b392-3a99853dfd94" (UID: "2afbb22b-8fa2-4695-b392-3a99853dfd94"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:16.866441 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.866411    5503 generic.go:356] "Generic (PLEG): container finished" podID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerID="8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4" exitCode=0
Oct 30 23:34:16.866597 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.866463    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"
Oct 30 23:34:16.866899 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.866589    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" event={"ID":"f059a50f-09a7-46bf-8ba8-de3332be620f","Type":"ContainerDied","Data":"8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4"}
Oct 30 23:34:16.867004 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.866950    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t" event={"ID":"f059a50f-09a7-46bf-8ba8-de3332be620f","Type":"ContainerDied","Data":"9ee071d835dedb69c48a8dc7fc0f1d4b0a8699cc3eb3921cb2e10cd07a4c7fad"}
Oct 30 23:34:16.867004 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.866983    5503 scope.go:122] "RemoveContainer" containerID="8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4"
Oct 30 23:34:16.868105 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.868080    5503 generic.go:356] "Generic (PLEG): container finished" podID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerID="ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d" exitCode=0
Oct 30 23:34:16.868161 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.868116    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"
Oct 30 23:34:16.868161 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.868152    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" event={"ID":"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f","Type":"ContainerDied","Data":"ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d"}
Oct 30 23:34:16.868243 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.868191    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p" event={"ID":"e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f","Type":"ContainerDied","Data":"9fe390b6ae828a2497b71940299635893cbe5ed13990d474d190c56109c7ff72"}
Oct 30 23:34:16.870071 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.870052    5503 generic.go:356] "Generic (PLEG): container finished" podID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerID="24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd" exitCode=0
Oct 30 23:34:16.870173 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.870081    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"
Oct 30 23:34:16.870173 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.870092    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" event={"ID":"2afbb22b-8fa2-4695-b392-3a99853dfd94","Type":"ContainerDied","Data":"24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd"}
Oct 30 23:34:16.870173 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.870129    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w" event={"ID":"2afbb22b-8fa2-4695-b392-3a99853dfd94","Type":"ContainerDied","Data":"44a4c876f5fd35a44b92aed54efa1c57d72aa1b63610c321f69bd5136a7d1ce0"}
Oct 30 23:34:16.912908 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.912887    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.912988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.912912    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.912988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.912928    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.912988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.912941    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.912988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.912957    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.912988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.912970    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7lzm\" (UniqueName: \"kubernetes.io/projected/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-kube-api-access-m7lzm\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.912988 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.912985    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.913192 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.912998    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.913192 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.913011    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9d2x7\" (UniqueName: \"kubernetes.io/projected/2afbb22b-8fa2-4695-b392-3a99853dfd94-kube-api-access-9d2x7\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.913192 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.913026    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.913192 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.913039    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2afbb22b-8fa2-4695-b392-3a99853dfd94-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.913192 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.913053    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.913192 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.913068    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls9g8\" (UniqueName: \"kubernetes.io/projected/f059a50f-09a7-46bf-8ba8-de3332be620f-kube-api-access-ls9g8\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.925166 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.925143    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"]
Oct 30 23:34:16.930354 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.930336    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-c7w8w"]
Oct 30 23:34:16.943110 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.943086    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"]
Oct 30 23:34:16.947978 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.947952    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-xfb4p"]
Oct 30 23:34:16.951617 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.951594    5503 scope.go:122] "RemoveContainer" containerID="8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4"
Oct 30 23:34:16.951921 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:34:16.951901    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4\": container with ID starting with 8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4 not found: ID does not exist" containerID="8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4"
Oct 30 23:34:16.951999 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.951935    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4"} err="failed to get container status \"8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4\": rpc error: code = NotFound desc = could not find container \"8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4\": container with ID starting with 8f06920e43002997ce0a0363e1c5ffb7fdc6989fd4695c2489d8802ae486c4a4 not found: ID does not exist"
Oct 30 23:34:16.951999 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.951959    5503 scope.go:122] "RemoveContainer" containerID="ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d"
Oct 30 23:34:16.960428 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.960331    5503 scope.go:122] "RemoveContainer" containerID="ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d"
Oct 30 23:34:16.960822 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:34:16.960806    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d\": container with ID starting with ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d not found: ID does not exist" containerID="ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d"
Oct 30 23:34:16.960872 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.960829    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d"} err="failed to get container status \"ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d\": rpc error: code = NotFound desc = could not find container \"ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d\": container with ID starting with ddaa7e139309016e84f7b8824c72e6eea0418694b03299e404f3a584950f323d not found: ID does not exist"
Oct 30 23:34:16.960872 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.960851    5503 scope.go:122] "RemoveContainer" containerID="24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd"
Oct 30 23:34:16.961429 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.961407    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"]
Oct 30 23:34:16.965374 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.965355    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-fbb5t"]
Oct 30 23:34:16.969197 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.969180    5503 scope.go:122] "RemoveContainer" containerID="24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd"
Oct 30 23:34:16.969462 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:34:16.969445    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd\": container with ID starting with 24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd not found: ID does not exist" containerID="24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd"
Oct 30 23:34:16.969512 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:16.969469    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd"} err="failed to get container status \"24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd\": rpc error: code = NotFound desc = could not find container \"24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd\": container with ID starting with 24be27f3c4a3059c10b444f2d1445b43d6a94443bed7fbcaf1d2447149cdaccd not found: ID does not exist"
Oct 30 23:34:17.077565 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.077541    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:34:17.114593 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.114562    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istiod-ca-cert\") pod \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") "
Oct 30 23:34:17.114750 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.114620    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-envoy\") pod \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") "
Oct 30 23:34:17.114750 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.114652    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-token\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-token\") pod \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") "
Oct 30 23:34:17.114750 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.114719    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-podinfo\") pod \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") "
Oct 30 23:34:17.114917 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.114759    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-kube-api-access-lq4rl\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-kube-api-access-lq4rl\") pod \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") "
Oct 30 23:34:17.114917 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.114816    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-certs\") pod \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") "
Oct 30 23:34:17.114917 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.114876    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-socket\") pod \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") "
Oct 30 23:34:17.115090 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.114926    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istiod-ca-cert" pod "a75d6804-7b72-4753-9ae3-0cbe3dadd26f" (UID: "a75d6804-7b72-4753-9ae3-0cbe3dadd26f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:34:17.115090 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.114975    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-credential-socket\") pod \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") "
Oct 30 23:34:17.115090 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.115019    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-data\") pod \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\" (UID: \"a75d6804-7b72-4753-9ae3-0cbe3dadd26f\") "
Oct 30 23:34:17.115260 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.115237    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-certs" pod "a75d6804-7b72-4753-9ae3-0cbe3dadd26f" (UID: "a75d6804-7b72-4753-9ae3-0cbe3dadd26f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:17.115342 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.115324    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.115399 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.115347    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.115489 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.115468    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-data" pod "a75d6804-7b72-4753-9ae3-0cbe3dadd26f" (UID: "a75d6804-7b72-4753-9ae3-0cbe3dadd26f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:17.115975 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.115808    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-credential-socket" pod "a75d6804-7b72-4753-9ae3-0cbe3dadd26f" (UID: "a75d6804-7b72-4753-9ae3-0cbe3dadd26f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:17.116792 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.116017    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-socket" pod "a75d6804-7b72-4753-9ae3-0cbe3dadd26f" (UID: "a75d6804-7b72-4753-9ae3-0cbe3dadd26f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:17.117586 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.117561    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-podinfo" pod "a75d6804-7b72-4753-9ae3-0cbe3dadd26f" (UID: "a75d6804-7b72-4753-9ae3-0cbe3dadd26f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:34:17.118021 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.118002    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-kube-api-access-lq4rl" pod "a75d6804-7b72-4753-9ae3-0cbe3dadd26f" (UID: "a75d6804-7b72-4753-9ae3-0cbe3dadd26f"). InnerVolumeSpecName "kube-api-access-lq4rl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:17.118096 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.118055    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-token" pod "a75d6804-7b72-4753-9ae3-0cbe3dadd26f" (UID: "a75d6804-7b72-4753-9ae3-0cbe3dadd26f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:17.118220 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.118204    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-envoy" pod "a75d6804-7b72-4753-9ae3-0cbe3dadd26f" (UID: "a75d6804-7b72-4753-9ae3-0cbe3dadd26f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:17.216639 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.216613    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.216639 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.216635    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.216803 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.216645    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq4rl\" (UniqueName: \"kubernetes.io/projected/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-kube-api-access-lq4rl\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.216803 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.216654    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.216803 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.216677    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.216803 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.216691    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.216803 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.216698    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a75d6804-7b72-4753-9ae3-0cbe3dadd26f-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.854974 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.854946    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" path="/var/lib/kubelet/pods/2afbb22b-8fa2-4695-b392-3a99853dfd94/volumes"
Oct 30 23:34:17.855345 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.855331    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" path="/var/lib/kubelet/pods/e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f/volumes"
Oct 30 23:34:17.855724 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.855712    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" path="/var/lib/kubelet/pods/f059a50f-09a7-46bf-8ba8-de3332be620f/volumes"
Oct 30 23:34:17.876875 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.876851    5503 generic.go:356] "Generic (PLEG): container finished" podID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerID="43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d" exitCode=0
Oct 30 23:34:17.876972 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.876899    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"
Oct 30 23:34:17.876972 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.876925    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" event={"ID":"a75d6804-7b72-4753-9ae3-0cbe3dadd26f","Type":"ContainerDied","Data":"43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d"}
Oct 30 23:34:17.877089 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.876971    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9" event={"ID":"a75d6804-7b72-4753-9ae3-0cbe3dadd26f","Type":"ContainerDied","Data":"1cdcd61f6f4e30cadc6839ae2940827bae09b21b578e27fe5ce0bab1fe25bf38"}
Oct 30 23:34:17.877089 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.877000    5503 scope.go:122] "RemoveContainer" containerID="43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d"
Oct 30 23:34:17.885437 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.885421    5503 scope.go:122] "RemoveContainer" containerID="43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d"
Oct 30 23:34:17.885747 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:34:17.885723    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d\": container with ID starting with 43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d not found: ID does not exist" containerID="43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d"
Oct 30 23:34:17.885827 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.885759    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d"} err="failed to get container status \"43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d\": rpc error: code = NotFound desc = could not find container \"43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d\": container with ID starting with 43fb60a43c08d868ac7fab876efc19da86a2c979d65e700d3dd0131165d5843d not found: ID does not exist"
Oct 30 23:34:17.899708 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.899687    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"]
Oct 30 23:34:17.904480 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:17.904461    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-7tpn9"]
Oct 30 23:34:19.855377 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:19.855344    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" path="/var/lib/kubelet/pods/a75d6804-7b72-4753-9ae3-0cbe3dadd26f/volumes"
Oct 30 23:34:22.526627 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:22.526590    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:23.315808 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:23.315775    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:34:33.315679 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:33.315588    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:34:36.414537 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.414515    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:34:36.452943 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.452880    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/86b1ad02-e855-4430-abd6-9435d5a0afa7-tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1ad02-e855-4430-abd6-9435d5a0afa7-tls-certs\") pod \"86b1ad02-e855-4430-abd6-9435d5a0afa7\" (UID: \"86b1ad02-e855-4430-abd6-9435d5a0afa7\") "
Oct 30 23:34:36.453066 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.452971    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86b1ad02-e855-4430-abd6-9435d5a0afa7-kube-api-access-4rd9h\" (UniqueName: \"kubernetes.io/projected/86b1ad02-e855-4430-abd6-9435d5a0afa7-kube-api-access-4rd9h\") pod \"86b1ad02-e855-4430-abd6-9435d5a0afa7\" (UID: \"86b1ad02-e855-4430-abd6-9435d5a0afa7\") "
Oct 30 23:34:36.455147 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.455122    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86b1ad02-e855-4430-abd6-9435d5a0afa7-kube-api-access-4rd9h" pod "86b1ad02-e855-4430-abd6-9435d5a0afa7" (UID: "86b1ad02-e855-4430-abd6-9435d5a0afa7"). InnerVolumeSpecName "kube-api-access-4rd9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:36.455265 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.455167    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86b1ad02-e855-4430-abd6-9435d5a0afa7-tls-certs" pod "86b1ad02-e855-4430-abd6-9435d5a0afa7" (UID: "86b1ad02-e855-4430-abd6-9435d5a0afa7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:34:36.553710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.553688    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/86b1ad02-e855-4430-abd6-9435d5a0afa7-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:36.553710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.553708    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rd9h\" (UniqueName: \"kubernetes.io/projected/86b1ad02-e855-4430-abd6-9435d5a0afa7-kube-api-access-4rd9h\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:36.958948 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.958925    5503 generic.go:356] "Generic (PLEG): container finished" podID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerID="1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1" exitCode=0
Oct 30 23:34:36.959052 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.958975    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"
Oct 30 23:34:36.959052 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.959011    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" event={"ID":"86b1ad02-e855-4430-abd6-9435d5a0afa7","Type":"ContainerDied","Data":"1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1"}
Oct 30 23:34:36.959132 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.959059    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5" event={"ID":"86b1ad02-e855-4430-abd6-9435d5a0afa7","Type":"ContainerDied","Data":"7a9deb8ac636d24d98db472460ba9dcc8b4ece21bdb9a1f57b5f8de6d8927877"}
Oct 30 23:34:36.959132 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.959082    5503 scope.go:122] "RemoveContainer" containerID="1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1"
Oct 30 23:34:36.967861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.967844    5503 scope.go:122] "RemoveContainer" containerID="1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1"
Oct 30 23:34:36.968117 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:34:36.968098    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1\": container with ID starting with 1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1 not found: ID does not exist" containerID="1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1"
Oct 30 23:34:36.968179 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.968124    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1"} err="failed to get container status \"1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1\": rpc error: code = NotFound desc = could not find container \"1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1\": container with ID starting with 1b2e0615c1225ad2659d76181f52f5433f84a217a9dcbe3eefb7ef0ce161baa1 not found: ID does not exist"
Oct 30 23:34:36.980319 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.980296    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"]
Oct 30 23:34:36.986136 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:36.986118    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-router-scheduler-69f7c679f8-x5lp5"]
Oct 30 23:34:37.600263 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:37.600243    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:37.855828 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:37.855782    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" path="/var/lib/kubelet/pods/86b1ad02-e855-4430-abd6-9435d5a0afa7/volumes"
Oct 30 23:34:42.177823 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:42.177795    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:42.280420 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:42.280394    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:45.388099 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:34:45.388065    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:34:45.953068 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:45.953040    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:46.084524 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084487    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"]
Oct 30 23:34:46.084872 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084857    5503 cpu_manager.go:414] "containerMap: removing container" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy"
Oct 30 23:34:46.084952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084873    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy"
Oct 30 23:34:46.084952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084886    5503 cpu_manager.go:414] "containerMap: removing container" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy"
Oct 30 23:34:46.084952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084891    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy"
Oct 30 23:34:46.084952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084899    5503 cpu_manager.go:414] "containerMap: removing container" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy"
Oct 30 23:34:46.084952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084903    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy"
Oct 30 23:34:46.084952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084917    5503 cpu_manager.go:414] "containerMap: removing container" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy"
Oct 30 23:34:46.084952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084921    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy"
Oct 30 23:34:46.084952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084935    5503 cpu_manager.go:414] "containerMap: removing container" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerName="main"
Oct 30 23:34:46.084952 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084939    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerName="main"
Oct 30 23:34:46.085392 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.084991    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="86b1ad02-e855-4430-abd6-9435d5a0afa7" containerName="main"
Oct 30 23:34:46.085392 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.085001    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="2afbb22b-8fa2-4695-b392-3a99853dfd94" containerName="istio-proxy"
Oct 30 23:34:46.085392 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.085007    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="a75d6804-7b72-4753-9ae3-0cbe3dadd26f" containerName="istio-proxy"
Oct 30 23:34:46.085392 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.085013    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="f059a50f-09a7-46bf-8ba8-de3332be620f" containerName="istio-proxy"
Oct 30 23:34:46.085392 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.085021    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9026bec-5c53-4ddb-8cf8-c6a92c58ba1f" containerName="istio-proxy"
Oct 30 23:34:46.089802 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.089776    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:34:46.100611 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.100585    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-kserve-self-signed-certs\""
Oct 30 23:34:46.100757 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.100707    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"openshift-service-ca.crt\""
Oct 30 23:34:46.101681 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.101643    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-epp-sa-dockercfg-kdvvv\""
Oct 30 23:34:46.101802 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.101720    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"default-dockercfg-46nkk\""
Oct 30 23:34:46.101802 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.101733    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"kube-root-ca.crt\""
Oct 30 23:34:46.102471 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.102441    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"]
Oct 30 23:34:46.155095 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.155061    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:46.225052 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.224984    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a02f10-8952-4a68-9953-010a176d3628-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24\" (UID: \"b9a02f10-8952-4a68-9953-010a176d3628\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:34:46.225052 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.225020    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsvxq\" (UniqueName: \"kubernetes.io/projected/b9a02f10-8952-4a68-9953-010a176d3628-kube-api-access-zsvxq\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24\" (UID: \"b9a02f10-8952-4a68-9953-010a176d3628\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:34:46.325969 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.325942    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a02f10-8952-4a68-9953-010a176d3628-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24\" (UID: \"b9a02f10-8952-4a68-9953-010a176d3628\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:34:46.326090 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.325975    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsvxq\" (UniqueName: \"kubernetes.io/projected/b9a02f10-8952-4a68-9953-010a176d3628-kube-api-access-zsvxq\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24\" (UID: \"b9a02f10-8952-4a68-9953-010a176d3628\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:34:46.328617 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.328588    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a02f10-8952-4a68-9953-010a176d3628-tls-certs\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24\" (UID: \"b9a02f10-8952-4a68-9953-010a176d3628\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:34:46.338449 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.338427    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsvxq\" (UniqueName: \"kubernetes.io/projected/b9a02f10-8952-4a68-9953-010a176d3628-kube-api-access-zsvxq\") pod \"prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24\" (UID: \"b9a02f10-8952-4a68-9953-010a176d3628\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:34:46.407011 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.406989    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:34:46.688609 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:46.688486    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"]
Oct 30 23:34:46.691408 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:34:46.691380    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9a02f10_8952_4a68_9953_010a176d3628.slice/crio-a01ad234b42e89a5bf3a0403955301b51b599bfe41ba8869c633d6c1406eb960 WatchSource:0}: Error finding container a01ad234b42e89a5bf3a0403955301b51b599bfe41ba8869c633d6c1406eb960: Status 404 returned error can't find the container with id a01ad234b42e89a5bf3a0403955301b51b599bfe41ba8869c633d6c1406eb960
Oct 30 23:34:47.004458 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:47.004378    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" event={"ID":"b9a02f10-8952-4a68-9953-010a176d3628","Type":"ContainerStarted","Data":"2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c"}
Oct 30 23:34:47.004458 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:47.004430    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" event={"ID":"b9a02f10-8952-4a68-9953-010a176d3628","Type":"ContainerStarted","Data":"a01ad234b42e89a5bf3a0403955301b51b599bfe41ba8869c633d6c1406eb960"}
Oct 30 23:34:47.004629 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:47.004470    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:34:47.337381 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:47.337320    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:58.989930 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:58.989893    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:34:59.593264 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:34:59.593220    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:35:11.575823 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.575771    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" podStartSLOduration=25.575754497 podStartE2EDuration="25.575754497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:34:47.064489298 +0000 UTC m=+4529.333570821" watchObservedRunningTime="2026-10-30 23:35:11.575754497 +0000 UTC m=+4553.844836022"
Oct 30 23:35:11.576513 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.576493    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"]
Oct 30 23:35:11.579888 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.579866    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.585172 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.585150    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-kxprg\""
Oct 30 23:35:11.591429 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.591400    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"]
Oct 30 23:35:11.595035 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.595004    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"]
Oct 30 23:35:11.595125 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.595097    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.607740 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.607719    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"]
Oct 30 23:35:11.715432 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715401    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/339f08b1-7307-4020-8d26-50f702cb8148-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.715555 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715439    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.715555 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715458    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.715555 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715507    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.715658 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715595    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.715658 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715615    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f56fdd-bb51-4c60-b597-323c0353b756-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.715658 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715637    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.715658 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715656    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.715844 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715757    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn7zb\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-kube-api-access-tn7zb\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.715844 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715779    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.715844 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715796    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.715844 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715810    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.715964 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715856    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.715964 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715931    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.716028 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715970    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqh8v\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-kube-api-access-mqh8v\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.716028 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.715987    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/339f08b1-7307-4020-8d26-50f702cb8148-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.716028 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.716007    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f56fdd-bb51-4c60-b597-323c0353b756-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.716126 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.716028    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.816803 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.816780    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.816909 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.816810    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/339f08b1-7307-4020-8d26-50f702cb8148-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.816909 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.816831    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.816909 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.816849    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817077 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.816962    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817077 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817003    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817077 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817032    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f56fdd-bb51-4c60-b597-323c0353b756-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817077 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817070    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817282 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817101    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817282 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817215    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817387 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817319    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817387 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817378    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817494 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817394    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817494 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817460    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817602 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817527    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn7zb\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-kube-api-access-tn7zb\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817602 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817565    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817602 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817570    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/339f08b1-7307-4020-8d26-50f702cb8148-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817602 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817592    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817897 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817616    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817897 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817652    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817897 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817761    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f56fdd-bb51-4c60-b597-323c0353b756-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.817897 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817788    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817897 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817829    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqh8v\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-kube-api-access-mqh8v\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817897 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817855    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/339f08b1-7307-4020-8d26-50f702cb8148-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.817897 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817891    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f56fdd-bb51-4c60-b597-323c0353b756-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.818168 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.817944    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.819021 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.818033    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.819021 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.818648    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.821405 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.821380    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.821658 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.821633    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f56fdd-bb51-4c60-b597-323c0353b756-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.821786 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.821764    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/339f08b1-7307-4020-8d26-50f702cb8148-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.826488 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.826437    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.829694 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.829652    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.830938 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.830911    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqh8v\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-kube-api-access-mqh8v\") pod \"openshift-ai-inference-openshift-default-86b669b864-t2dz9\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:11.831041 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.830922    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.831103 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.831068    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn7zb\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-kube-api-access-tn7zb\") pod \"openshift-ai-inference-openshift-default-86b669b864-2c7vh\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.894473 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.894454    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:11.907605 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:11.907587    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:12.179444 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:12.179421    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"]
Oct 30 23:35:12.181373 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:35:12.181341    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f56fdd_bb51_4c60_b597_323c0353b756.slice/crio-a72b2870b45370fc64e3585cdc4aec4d0680bdb1e70dc8e255231b08aad2a7b5 WatchSource:0}: Error finding container a72b2870b45370fc64e3585cdc4aec4d0680bdb1e70dc8e255231b08aad2a7b5: Status 404 returned error can't find the container with id a72b2870b45370fc64e3585cdc4aec4d0680bdb1e70dc8e255231b08aad2a7b5
Oct 30 23:35:12.183230 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:12.183202    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:12.183308 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:12.183264    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:12.183308 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:12.183292    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:12.187503 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:12.187472    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"]
Oct 30 23:35:12.190696 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:35:12.190650    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod339f08b1_7307_4020_8d26_50f702cb8148.slice/crio-3938b31bac519726d3401c0a74dcb30c1be95578fc8ebaf52673c4642174f009 WatchSource:0}: Error finding container 3938b31bac519726d3401c0a74dcb30c1be95578fc8ebaf52673c4642174f009: Status 404 returned error can't find the container with id 3938b31bac519726d3401c0a74dcb30c1be95578fc8ebaf52673c4642174f009
Oct 30 23:35:12.192617 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:12.192596    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:12.192713 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:12.192659    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:12.192759 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:12.192738    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:12.775562 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:12.775526    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:35:13.147986 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.147940    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" event={"ID":"62f56fdd-bb51-4c60-b597-323c0353b756","Type":"ContainerStarted","Data":"681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c"}
Oct 30 23:35:13.148187 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.148022    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" event={"ID":"62f56fdd-bb51-4c60-b597-323c0353b756","Type":"ContainerStarted","Data":"a72b2870b45370fc64e3585cdc4aec4d0680bdb1e70dc8e255231b08aad2a7b5"}
Oct 30 23:35:13.149887 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.149862    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" event={"ID":"339f08b1-7307-4020-8d26-50f702cb8148","Type":"ContainerStarted","Data":"7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05"}
Oct 30 23:35:13.149994 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.149900    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" event={"ID":"339f08b1-7307-4020-8d26-50f702cb8148","Type":"ContainerStarted","Data":"3938b31bac519726d3401c0a74dcb30c1be95578fc8ebaf52673c4642174f009"}
Oct 30 23:35:13.177052 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.177010    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" podStartSLOduration=2.176998471 podStartE2EDuration="2.176998471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:35:13.174886513 +0000 UTC m=+4555.443968039" watchObservedRunningTime="2026-10-30 23:35:13.176998471 +0000 UTC m=+4555.446079996"
Oct 30 23:35:13.902692 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.900492    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:13.902692 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.900602    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-2c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 23:35:13.902692 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.900645    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" podUID="62f56fdd-bb51-4c60-b597-323c0353b756" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 23:35:13.911234 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.911211    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:13.912911 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.912887    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-t2dz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 23:35:13.913074 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:13.913049    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" podUID="339f08b1-7307-4020-8d26-50f702cb8148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 23:35:14.899066 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:14.899029    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-2c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 23:35:14.899337 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:14.899310    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" podUID="62f56fdd-bb51-4c60-b597-323c0353b756" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 23:35:14.909499 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:14.909474    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-t2dz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 23:35:14.910709 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:14.910648    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" podUID="339f08b1-7307-4020-8d26-50f702cb8148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 23:35:15.388791 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:35:15.388756    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:35:15.902693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:15.898775    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-2c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 23:35:15.902693 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:15.898841    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" podUID="62f56fdd-bb51-4c60-b597-323c0353b756" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 23:35:15.910690 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:15.910458    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-t2dz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 23:35:15.910690 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:15.910516    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" podUID="339f08b1-7307-4020-8d26-50f702cb8148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 23:35:16.899038 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:16.899001    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-2c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 23:35:16.899362 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:16.899333    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" podUID="62f56fdd-bb51-4c60-b597-323c0353b756" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 23:35:16.908146 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:16.908117    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-t2dz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 23:35:16.908342 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:16.908319    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" podUID="339f08b1-7307-4020-8d26-50f702cb8148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 23:35:17.901191 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:17.901155    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-2c7vh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused" start-of-body=
Oct 30 23:35:17.901835 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:17.901803    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" podUID="62f56fdd-bb51-4c60-b597-323c0353b756" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.85:15021/healthz/ready\": dial tcp 10.133.0.85:15021: connect: connection refused"
Oct 30 23:35:17.917107 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:17.917069    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-t2dz9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused" start-of-body=
Oct 30 23:35:17.917371 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:17.917339    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" podUID="339f08b1-7307-4020-8d26-50f702cb8148" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.86:15021/healthz/ready\": dial tcp 10.133.0.86:15021: connect: connection refused"
Oct 30 23:35:18.014455 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:18.014424    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:35:18.034552 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:18.034484    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" podStartSLOduration=7.034464484 podStartE2EDuration="7.034464484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:35:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:35:13.215965581 +0000 UTC m=+4555.485047105" watchObservedRunningTime="2026-10-30 23:35:18.034464484 +0000 UTC m=+4560.303546008"
Oct 30 23:35:18.898616 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:18.898547    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:18.898884 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:18.898860    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:18.899508 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:18.899491    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:35:18.911833 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:18.911814    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:18.912158 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:18.912050    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:18.912576 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:18.912561    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:35:26.627597 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.627562    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"]
Oct 30 23:35:26.631097 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.631075    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.640815 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.640789    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"]
Oct 30 23:35:26.646447 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.646425    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"]
Oct 30 23:35:26.646559 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.646528    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.663449 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.663425    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"]
Oct 30 23:35:26.756800 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.756771    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z57vx\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-kube-api-access-z57vx\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.756939 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.756801    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.756939 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.756829    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.756939 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.756912    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.757075 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.756966    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.757075 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757005    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.757075 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757026    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993bc0f1-0397-46de-ab78-8f014b5b39a4-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.757075 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757041    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjtzn\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-kube-api-access-gjtzn\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.757075 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757055    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.757282 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757151    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.757282 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757169    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.757282 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757196    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.757282 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757213    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.757282 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757257    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.757493 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757298    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.757493 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757344    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.757493 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757409    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.757493 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.757429    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.857927 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.857899    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858032 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.857930    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858032 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.857958    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993bc0f1-0397-46de-ab78-8f014b5b39a4-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858032 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.857981    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjtzn\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-kube-api-access-gjtzn\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858032 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858005    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858042    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858065    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858106    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858132    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858158    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858185    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858208    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858246    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858277    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858335    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z57vx\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-kube-api-access-z57vx\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858361    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858405    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858440    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858443    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858444    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.858777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858506    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858728    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993bc0f1-0397-46de-ab78-8f014b5b39a4-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.858777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858753    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.859215 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.858819    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.859215 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.859026    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.859215 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.859182    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.859370 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.859238    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.859370 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.859251    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.860637 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.860614    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.860904 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.860883    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.860948 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.860885    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.861591 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.861569    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.871485 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.871465    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.871614 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.871586    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.871818 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.871802    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjtzn\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-kube-api-access-gjtzn\") pod \"openshift-ai-inference-openshift-default-86b669b864-wvd5p\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:26.872061 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.872038    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z57vx\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-kube-api-access-z57vx\") pod \"openshift-ai-inference-openshift-default-86b669b864-5cx6f\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.946502 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.946447    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:26.959391 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:26.959353    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:27.232488 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:27.232458    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"]
Oct 30 23:35:27.234225 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:35:27.234199    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod993bc0f1_0397_46de_ab78_8f014b5b39a4.slice/crio-6b179edf6db10923391c3d72b822129354fda0484a350c11f9ce9d3c441ef144 WatchSource:0}: Error finding container 6b179edf6db10923391c3d72b822129354fda0484a350c11f9ce9d3c441ef144: Status 404 returned error can't find the container with id 6b179edf6db10923391c3d72b822129354fda0484a350c11f9ce9d3c441ef144
Oct 30 23:35:27.236219 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:27.236189    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:27.236279 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:27.236250    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:27.236279 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:27.236275    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:27.256353 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:27.256331    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"]
Oct 30 23:35:27.268689 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:35:27.268652    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37905f9c_9cf9_42e1_921f_33db04ec8ab9.slice/crio-86aa8a303dc2e59a386647b9aa4fcc0a27696800dd3ccb7da4b108e9853a6947 WatchSource:0}: Error finding container 86aa8a303dc2e59a386647b9aa4fcc0a27696800dd3ccb7da4b108e9853a6947: Status 404 returned error can't find the container with id 86aa8a303dc2e59a386647b9aa4fcc0a27696800dd3ccb7da4b108e9853a6947
Oct 30 23:35:27.270722 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:27.270689    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:27.270810 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:27.270765    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:27.270850 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:27.270810    5503 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:28.247304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.247189    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" event={"ID":"37905f9c-9cf9-42e1-921f-33db04ec8ab9","Type":"ContainerStarted","Data":"3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155"}
Oct 30 23:35:28.247304 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.247263    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" event={"ID":"37905f9c-9cf9-42e1-921f-33db04ec8ab9","Type":"ContainerStarted","Data":"86aa8a303dc2e59a386647b9aa4fcc0a27696800dd3ccb7da4b108e9853a6947"}
Oct 30 23:35:28.249567 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.249539    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" event={"ID":"993bc0f1-0397-46de-ab78-8f014b5b39a4","Type":"ContainerStarted","Data":"95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85"}
Oct 30 23:35:28.249714 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.249585    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" event={"ID":"993bc0f1-0397-46de-ab78-8f014b5b39a4","Type":"ContainerStarted","Data":"6b179edf6db10923391c3d72b822129354fda0484a350c11f9ce9d3c441ef144"}
Oct 30 23:35:28.273875 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.273811    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" podStartSLOduration=2.273790125 podStartE2EDuration="2.273790125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:35:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:35:28.270938356 +0000 UTC m=+4570.540019893" watchObservedRunningTime="2026-10-30 23:35:28.273790125 +0000 UTC m=+4570.542871648"
Oct 30 23:35:28.947570 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.947519    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:28.949122 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.949092    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5cx6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 23:35:28.949251 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.949148    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" podUID="993bc0f1-0397-46de-ab78-8f014b5b39a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 23:35:28.959701 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.959655    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:28.960824 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.960802    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-wvd5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 23:35:28.960900 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:28.960841    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" podUID="37905f9c-9cf9-42e1-921f-33db04ec8ab9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 23:35:29.947251 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:29.947217    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5cx6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 23:35:29.947684 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:29.947273    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" podUID="993bc0f1-0397-46de-ab78-8f014b5b39a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 23:35:29.959782 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:29.959755    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-wvd5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 23:35:29.959902 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:29.959811    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" podUID="37905f9c-9cf9-42e1-921f-33db04ec8ab9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 23:35:30.947257 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:30.947216    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5cx6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 23:35:30.947989 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:30.947954    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" podUID="993bc0f1-0397-46de-ab78-8f014b5b39a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 23:35:30.959823 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:30.959797    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-wvd5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 23:35:30.959951 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:30.959839    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" podUID="37905f9c-9cf9-42e1-921f-33db04ec8ab9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 23:35:31.947257 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:31.947220    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5cx6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 23:35:31.947534 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:31.947509    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" podUID="993bc0f1-0397-46de-ab78-8f014b5b39a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 23:35:31.960575 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:31.960545    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-wvd5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 23:35:31.960796 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:31.960774    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" podUID="37905f9c-9cf9-42e1-921f-33db04ec8ab9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 23:35:32.590199 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:32.590164    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:35:32.947384 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:32.947351    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-5cx6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused" start-of-body=
Oct 30 23:35:32.947562 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:32.947415    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" podUID="993bc0f1-0397-46de-ab78-8f014b5b39a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.87:15021/healthz/ready\": dial tcp 10.133.0.87:15021: connect: connection refused"
Oct 30 23:35:32.960614 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:32.960580    5503 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-wvd5p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused" start-of-body=
Oct 30 23:35:32.960758 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:32.960641    5503 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" podUID="37905f9c-9cf9-42e1-921f-33db04ec8ab9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.88:15021/healthz/ready\": dial tcp 10.133.0.88:15021: connect: connection refused"
Oct 30 23:35:33.950756 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:33.950728    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:33.951128 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:33.951073    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:33.951811 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:33.951796    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:35:33.963097 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:33.963074    5503 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:33.963320 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:33.963292    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:33.963888 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:33.963872    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:35:33.979649 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:33.979606    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" podStartSLOduration=7.979596826 podStartE2EDuration="7.979596826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:35:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:35:28.309738296 +0000 UTC m=+4570.578819819" watchObservedRunningTime="2026-10-30 23:35:33.979596826 +0000 UTC m=+4576.248678349"
Oct 30 23:35:45.389457 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:35:45.389419    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:35:47.654337 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:35:47.654294    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:36:07.433218 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:36:07.433179    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:36:12.800725 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:36:12.800685    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:36:15.390451 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:36:15.390408    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:36:26.173465 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:36:26.173428    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:36:26.330558 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:36:26.330526    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:36:26.476084 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:36:26.475985    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:36:26.590491 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:36:26.590461    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:36:32.780722 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:36:32.780655    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:36:36.013872 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:36:36.013838    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:36:45.388192 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:36:45.388159    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:36:47.714215 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:36:47.714179    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:37:07.360110 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:07.360060    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"]
Oct 30 23:37:07.362794 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:07.362765    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:37:07.481923 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:07.481831    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:37:08.010496 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:08.010455    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" podUID="b9a02f10-8952-4a68-9953-010a176d3628" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:37:12.895198 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:12.895151    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:37:15.389420 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:37:15.389378    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:37:18.010098 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:18.010055    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" podUID="b9a02f10-8952-4a68-9953-010a176d3628" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:37:22.380249 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:22.380185    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" podUID="b9a02f10-8952-4a68-9953-010a176d3628" containerName="main" containerID="cri-o://2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c" gracePeriod=45
Oct 30 23:37:28.010832 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:28.010747    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" podUID="b9a02f10-8952-4a68-9953-010a176d3628" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:37:28.011239 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:28.010858    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:37:32.721971 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:32.721934    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:37:38.010375 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:38.010335    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" podUID="b9a02f10-8952-4a68-9953-010a176d3628" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:37:45.390215 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:37:45.390174    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:37:47.765307 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:47.765273    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:37:48.010316 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:48.010275    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" podUID="b9a02f10-8952-4a68-9953-010a176d3628" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:37:52.637813 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.637784    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:37:52.735512 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.735413    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b9a02f10-8952-4a68-9953-010a176d3628-tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a02f10-8952-4a68-9953-010a176d3628-tls-certs\") pod \"b9a02f10-8952-4a68-9953-010a176d3628\" (UID: \"b9a02f10-8952-4a68-9953-010a176d3628\") "
Oct 30 23:37:52.735512 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.735476    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9a02f10-8952-4a68-9953-010a176d3628-kube-api-access-zsvxq\" (UniqueName: \"kubernetes.io/projected/b9a02f10-8952-4a68-9953-010a176d3628-kube-api-access-zsvxq\") pod \"b9a02f10-8952-4a68-9953-010a176d3628\" (UID: \"b9a02f10-8952-4a68-9953-010a176d3628\") "
Oct 30 23:37:52.737869 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.737836    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b9a02f10-8952-4a68-9953-010a176d3628-tls-certs" pod "b9a02f10-8952-4a68-9953-010a176d3628" (UID: "b9a02f10-8952-4a68-9953-010a176d3628"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:37:52.737998 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.737878    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9a02f10-8952-4a68-9953-010a176d3628-kube-api-access-zsvxq" pod "b9a02f10-8952-4a68-9953-010a176d3628" (UID: "b9a02f10-8952-4a68-9953-010a176d3628"). InnerVolumeSpecName "kube-api-access-zsvxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:37:52.836431 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.836376    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9a02f10-8952-4a68-9953-010a176d3628-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:37:52.836431 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.836424    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsvxq\" (UniqueName: \"kubernetes.io/projected/b9a02f10-8952-4a68-9953-010a176d3628-kube-api-access-zsvxq\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:37:52.966184 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.966148    5503 generic.go:356] "Generic (PLEG): container finished" podID="b9a02f10-8952-4a68-9953-010a176d3628" containerID="2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c" exitCode=0
Oct 30 23:37:52.966184 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.966194    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"
Oct 30 23:37:52.966463 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.966201    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" event={"ID":"b9a02f10-8952-4a68-9953-010a176d3628","Type":"ContainerDied","Data":"2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c"}
Oct 30 23:37:52.966463 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.966257    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24" event={"ID":"b9a02f10-8952-4a68-9953-010a176d3628","Type":"ContainerDied","Data":"a01ad234b42e89a5bf3a0403955301b51b599bfe41ba8869c633d6c1406eb960"}
Oct 30 23:37:52.966463 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.966286    5503 scope.go:122] "RemoveContainer" containerID="2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c"
Oct 30 23:37:52.976251 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.976205    5503 scope.go:122] "RemoveContainer" containerID="2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c"
Oct 30 23:37:52.976514 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:37:52.976493    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c\": container with ID starting with 2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c not found: ID does not exist" containerID="2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c"
Oct 30 23:37:52.976557 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.976526    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c"} err="failed to get container status \"2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c\": rpc error: code = NotFound desc = could not find container \"2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c\": container with ID starting with 2dbdea69bbb32989a1ed139949eded4d15523990829be33dd15e71536a338d8c not found: ID does not exist"
Oct 30 23:37:52.994411 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.994320    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"]
Oct 30 23:37:52.994411 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:52.994391    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-router-scheduler-89ccd5fbb-zrz24"]
Oct 30 23:37:53.855271 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:53.855230    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9a02f10-8952-4a68-9953-010a176d3628" path="/var/lib/kubelet/pods/b9a02f10-8952-4a68-9953-010a176d3628/volumes"
Oct 30 23:37:53.920906 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:37:53.920872    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:38:12.405741 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:38:12.405696    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:38:12.508318 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:38:12.508280    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:38:12.882372 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:38:12.882343    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:38:12.984807 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:38:12.984772    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:38:15.388861 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:38:15.388821    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:38:19.454512 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:38:19.454482    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:38:25.247365 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:38:25.247315    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:38:25.355171 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:38:25.355140    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:38:32.750042 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:38:32.750004    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:38:45.389051 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:38:45.389008    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:38:47.881884 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:38:47.881849    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:02.722225 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:02.722183    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:02.832147 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:02.832105    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:02.932456 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:02.932424    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:09.286861 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.286825    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:09.434923 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.434874    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:09.547758 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.547653    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"]
Oct 30 23:39:09.548326 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.548307    5503 cpu_manager.go:414] "containerMap: removing container" podUID="b9a02f10-8952-4a68-9953-010a176d3628" containerName="main"
Oct 30 23:39:09.548383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.548330    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9a02f10-8952-4a68-9953-010a176d3628" containerName="main"
Oct 30 23:39:09.548447 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.548435    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9a02f10-8952-4a68-9953-010a176d3628" containerName="main"
Oct 30 23:39:09.551931 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.551910    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:09.558177 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.558153    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-epp-sa-dockercfg-whwj6\""
Oct 30 23:39:09.560402 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.558512    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Oct 30 23:39:09.560402 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.560137    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Oct 30 23:39:09.561280 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.561257    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-x4nkf\""
Oct 30 23:39:09.561392 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.561347    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Oct 30 23:39:09.562419 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.562397    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"]
Oct 30 23:39:09.627359 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.627335    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:09.648157 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.648133    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28093edf-3a2d-4441-a913-e809927b4cf2-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb\" (UID: \"28093edf-3a2d-4441-a913-e809927b4cf2\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:09.648252 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.648196    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg6w8\" (UniqueName: \"kubernetes.io/projected/28093edf-3a2d-4441-a913-e809927b4cf2-kube-api-access-xg6w8\") pod \"rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb\" (UID: \"28093edf-3a2d-4441-a913-e809927b4cf2\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:09.748809 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.748770    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28093edf-3a2d-4441-a913-e809927b4cf2-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb\" (UID: \"28093edf-3a2d-4441-a913-e809927b4cf2\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:09.748989 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.748870    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg6w8\" (UniqueName: \"kubernetes.io/projected/28093edf-3a2d-4441-a913-e809927b4cf2-kube-api-access-xg6w8\") pod \"rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb\" (UID: \"28093edf-3a2d-4441-a913-e809927b4cf2\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:09.751512 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.751482    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28093edf-3a2d-4441-a913-e809927b4cf2-tls-certs\") pod \"rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb\" (UID: \"28093edf-3a2d-4441-a913-e809927b4cf2\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:09.762549 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.762524    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg6w8\" (UniqueName: \"kubernetes.io/projected/28093edf-3a2d-4441-a913-e809927b4cf2-kube-api-access-xg6w8\") pod \"rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb\" (UID: \"28093edf-3a2d-4441-a913-e809927b4cf2\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:09.867339 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:09.867316    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:10.145530 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:10.145504    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"]
Oct 30 23:39:10.147624 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:39:10.147587    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28093edf_3a2d_4441_a913_e809927b4cf2.slice/crio-a4c8048b083bba76076e66327dd3074b6f2b1de3f3908d70803c8ca5850fe91b WatchSource:0}: Error finding container a4c8048b083bba76076e66327dd3074b6f2b1de3f3908d70803c8ca5850fe91b: Status 404 returned error can't find the container with id a4c8048b083bba76076e66327dd3074b6f2b1de3f3908d70803c8ca5850fe91b
Oct 30 23:39:10.348771 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:10.348709    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" event={"ID":"28093edf-3a2d-4441-a913-e809927b4cf2","Type":"ContainerStarted","Data":"c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3"}
Oct 30 23:39:10.349200 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:10.348789    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:10.349200 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:10.348808    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" event={"ID":"28093edf-3a2d-4441-a913-e809927b4cf2","Type":"ContainerStarted","Data":"a4c8048b083bba76076e66327dd3074b6f2b1de3f3908d70803c8ca5850fe91b"}
Oct 30 23:39:13.097333 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:13.097283    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:15.160604 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:15.160563    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:15.275012 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:15.274981    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:15.495752 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:39:15.493957    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:39:22.614601 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:22.614560    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:32.829785 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:32.829747    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:41.364742 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:41.364711    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:39:41.385086 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:41.385040    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" podStartSLOduration=32.385027838 podStartE2EDuration="32.385027838s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:39:10.396946722 +0000 UTC m=+4792.666028282" watchObservedRunningTime="2026-10-30 23:39:41.385027838 +0000 UTC m=+4823.654109360"
Oct 30 23:39:45.441606 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:39:45.441568    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:39:50.279267 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:50.279223    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"]
Oct 30 23:39:50.335983 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:50.335952    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:50.451365 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:50.451330    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:39:51.363944 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:39:51.363906    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:40:01.364044 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:01.364013    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:40:05.300539 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:05.300472    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" containerName="main" containerID="cri-o://c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3" gracePeriod=45
Oct 30 23:40:11.364415 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:11.364375    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:40:11.364796 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:11.364480    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:40:13.176824 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:13.176778    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:40:15.389073 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:40:15.389037    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:40:19.309221 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:19.309184    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:40:21.363933 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:21.363893    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:40:31.364226 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:31.364143    5503 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" containerName="main" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 23:40:32.866891 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:32.866851    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:40:35.552066 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.552040    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:40:35.712877 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.712835    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/28093edf-3a2d-4441-a913-e809927b4cf2-tls-certs\" (UniqueName: \"kubernetes.io/secret/28093edf-3a2d-4441-a913-e809927b4cf2-tls-certs\") pod \"28093edf-3a2d-4441-a913-e809927b4cf2\" (UID: \"28093edf-3a2d-4441-a913-e809927b4cf2\") "
Oct 30 23:40:35.713068 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.712898    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28093edf-3a2d-4441-a913-e809927b4cf2-kube-api-access-xg6w8\" (UniqueName: \"kubernetes.io/projected/28093edf-3a2d-4441-a913-e809927b4cf2-kube-api-access-xg6w8\") pod \"28093edf-3a2d-4441-a913-e809927b4cf2\" (UID: \"28093edf-3a2d-4441-a913-e809927b4cf2\") "
Oct 30 23:40:35.715249 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.715220    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28093edf-3a2d-4441-a913-e809927b4cf2-tls-certs" pod "28093edf-3a2d-4441-a913-e809927b4cf2" (UID: "28093edf-3a2d-4441-a913-e809927b4cf2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:40:35.715249 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.715236    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28093edf-3a2d-4441-a913-e809927b4cf2-kube-api-access-xg6w8" pod "28093edf-3a2d-4441-a913-e809927b4cf2" (UID: "28093edf-3a2d-4441-a913-e809927b4cf2"). InnerVolumeSpecName "kube-api-access-xg6w8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:40:35.813960 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.813923    5503 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/28093edf-3a2d-4441-a913-e809927b4cf2-tls-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:40:35.813960 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.813957    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xg6w8\" (UniqueName: \"kubernetes.io/projected/28093edf-3a2d-4441-a913-e809927b4cf2-kube-api-access-xg6w8\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:40:35.824329 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.824293    5503 generic.go:356] "Generic (PLEG): container finished" podID="28093edf-3a2d-4441-a913-e809927b4cf2" containerID="c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3" exitCode=0
Oct 30 23:40:35.824494 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.824340    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"
Oct 30 23:40:35.824494 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.824378    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" event={"ID":"28093edf-3a2d-4441-a913-e809927b4cf2","Type":"ContainerDied","Data":"c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3"}
Oct 30 23:40:35.824494 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.824429    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb" event={"ID":"28093edf-3a2d-4441-a913-e809927b4cf2","Type":"ContainerDied","Data":"a4c8048b083bba76076e66327dd3074b6f2b1de3f3908d70803c8ca5850fe91b"}
Oct 30 23:40:35.824494 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.824454    5503 scope.go:122] "RemoveContainer" containerID="c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3"
Oct 30 23:40:35.834408 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.834383    5503 scope.go:122] "RemoveContainer" containerID="c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3"
Oct 30 23:40:35.834736 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:40:35.834716    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3\": container with ID starting with c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3 not found: ID does not exist" containerID="c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3"
Oct 30 23:40:35.834795 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.834746    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3"} err="failed to get container status \"c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3\": rpc error: code = NotFound desc = could not find container \"c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3\": container with ID starting with c3fc60cec9a46b73fe3b63590d7480626bcdf0cc5ca9a4064f0260a45afef8c3 not found: ID does not exist"
Oct 30 23:40:35.847911 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.847882    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"]
Oct 30 23:40:35.855340 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:35.855309    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-router-scheduler-7455cbc656-kpjsb"]
Oct 30 23:40:37.856327 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:37.856290    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" path="/var/lib/kubelet/pods/28093edf-3a2d-4441-a913-e809927b4cf2/volumes"
Oct 30 23:40:40.377603 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:40.377562    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:40:45.390333 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:40:45.390292    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:40:46.761822 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:46.761787    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:40:52.618525 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:52.618494    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:40:52.736886 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:52.736854    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:40:52.899354 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:52.899271    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:40:55.014847 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:55.014812    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:40:55.201820 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:40:55.201788    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:41:13.205803 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:41:13.205760    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:41:15.394803 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:41:15.394760    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:41:32.904208 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:41:32.904172    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:41:45.389692 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:41:45.389635    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:41:52.956403 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:41:52.956363    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:42:13.262730 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:42:13.262695    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:42:15.389880 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:42:15.389844    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:42:33.020982 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:42:33.020948    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:42:39.091200 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:42:39.091165    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:42:45.391098 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:42:45.391059    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:42:53.022958 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:42:53.022920    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:42:58.832941 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:42:58.832899    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:42:58.980402 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:42:58.980364    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:43:10.468178 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:10.468146    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-wr97s_40a5acae-35b7-4ac0-911b-7699d0127186/discovery/0.log"
Oct 30 23:43:10.483828 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:10.483802    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-2c7vh_62f56fdd-bb51-4c60-b597-323c0353b756/istio-proxy/0.log"
Oct 30 23:43:10.505559 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:10.505533    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-5cx6f_993bc0f1-0397-46de-ab78-8f014b5b39a4/istio-proxy/0.log"
Oct 30 23:43:10.727891 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:10.727801    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-t2dz9_339f08b1-7307-4020-8d26-50f702cb8148/istio-proxy/0.log"
Oct 30 23:43:10.749370 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:10.749337    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-wvd5p_37905f9c-9cf9-42e1-921f-33db04ec8ab9/istio-proxy/0.log"
Oct 30 23:43:11.978579 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:11.978546    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-wr97s_40a5acae-35b7-4ac0-911b-7699d0127186/discovery/0.log"
Oct 30 23:43:11.997711 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:11.997659    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-2c7vh_62f56fdd-bb51-4c60-b597-323c0353b756/istio-proxy/0.log"
Oct 30 23:43:12.018371 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:12.018347    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-5cx6f_993bc0f1-0397-46de-ab78-8f014b5b39a4/istio-proxy/0.log"
Oct 30 23:43:12.135629 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:12.135596    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-t2dz9_339f08b1-7307-4020-8d26-50f702cb8148/istio-proxy/0.log"
Oct 30 23:43:12.155651 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:12.155626    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-wvd5p_37905f9c-9cf9-42e1-921f-33db04ec8ab9/istio-proxy/0.log"
Oct 30 23:43:13.138934 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:13.138898    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-557c8f8584-6kpnv_b73480bd-92cc-4f2d-a4ed-c794d61edab0/authorino/0.log"
Oct 30 23:43:13.153550 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:13.153515    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-66498fdf6c-b8722_66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e/manager/0.log"
Oct 30 23:43:14.368727 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:14.368692    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:43:15.389399 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:43:15.389358    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:43:15.700903 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:15.700814    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:43:16.031025 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:16.030947    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:43:16.149929 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:16.149895    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:43:19.151463 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:19.151431    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hgsqv_1ae3621a-b431-4675-8617-37461922d241/global-pull-secret-syncer/0.log"
Oct 30 23:43:19.345283 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:19.345248    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log"
Oct 30 23:43:19.503696 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:19.503580    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-97.us-east-2.compute.internal_e09f6b9a488f34d09a2deb5cc4fd598a/haproxy/0.log"
Oct 30 23:43:22.112065 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:22.112032    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:43:22.217798 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:22.217764    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:43:45.389715 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:43:45.389480    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:43:52.447430 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:52.447391    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-557c8f8584-6kpnv_b73480bd-92cc-4f2d-a4ed-c794d61edab0/authorino/0.log"
Oct 30 23:43:52.485777 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:52.485745    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-66498fdf6c-b8722_66d6e6c1-1a6a-4b1f-b7c8-04aabd0bf00e/manager/0.log"
Oct 30 23:43:53.036005 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.035969    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:43:53.146306 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.146271    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:43:53.530236 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.530201    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19fca836-f67e-48a6-a9b6-3060f0261e51/alertmanager/0.log"
Oct 30 23:43:53.566541 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.566513    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19fca836-f67e-48a6-a9b6-3060f0261e51/config-reloader/0.log"
Oct 30 23:43:53.598516 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.598490    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19fca836-f67e-48a6-a9b6-3060f0261e51/kube-rbac-proxy-web/0.log"
Oct 30 23:43:53.633598 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.633566    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19fca836-f67e-48a6-a9b6-3060f0261e51/kube-rbac-proxy/0.log"
Oct 30 23:43:53.664432 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.664403    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19fca836-f67e-48a6-a9b6-3060f0261e51/kube-rbac-proxy-metric/0.log"
Oct 30 23:43:53.695881 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.695856    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19fca836-f67e-48a6-a9b6-3060f0261e51/prom-label-proxy/0.log"
Oct 30 23:43:53.728547 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.728514    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_19fca836-f67e-48a6-a9b6-3060f0261e51/init-config-reloader/0.log"
Oct 30 23:43:53.924926 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.924893    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5f5784d99-gnbws_8db49c21-a7b4-4fc6-8342-95243844e22f/metrics-server/0.log"
Oct 30 23:43:53.966019 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:53.965983    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-249ck_9115c245-4790-4124-a435-24b667fc1744/monitoring-plugin/0.log"
Oct 30 23:43:54.245780 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:54.245697    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jxc4l_2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/node-exporter/0.log"
Oct 30 23:43:54.291111 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:54.291075    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jxc4l_2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/kube-rbac-proxy/0.log"
Oct 30 23:43:54.328482 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:54.328458    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jxc4l_2ecaf39f-ddd9-488f-9c8a-8fc7664eb2f2/init-textfile/0.log"
Oct 30 23:43:54.852120 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:54.852089    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-9dp62_a410d5a1-3091-405c-a1b2-9d790b77b458/prometheus-operator/0.log"
Oct 30 23:43:54.884237 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:54.884211    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-9dp62_a410d5a1-3091-405c-a1b2-9d790b77b458/kube-rbac-proxy/0.log"
Oct 30 23:43:54.987174 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:54.987143    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56c84f9555-d5rqg_9701f603-d98d-49b5-adc7-3550bdab41d2/telemeter-client/0.log"
Oct 30 23:43:55.016778 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.016740    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56c84f9555-d5rqg_9701f603-d98d-49b5-adc7-3550bdab41d2/reload/0.log"
Oct 30 23:43:55.049527 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.049502    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-56c84f9555-d5rqg_9701f603-d98d-49b5-adc7-3550bdab41d2/kube-rbac-proxy/0.log"
Oct 30 23:43:55.084313 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.084287    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6776c6b778-2gtz5_c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/thanos-query/0.log"
Oct 30 23:43:55.122656 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.122577    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6776c6b778-2gtz5_c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/kube-rbac-proxy-web/0.log"
Oct 30 23:43:55.173205 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.173178    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6776c6b778-2gtz5_c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/kube-rbac-proxy/0.log"
Oct 30 23:43:55.232999 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.232968    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6776c6b778-2gtz5_c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/prom-label-proxy/0.log"
Oct 30 23:43:55.270526 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.270496    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6776c6b778-2gtz5_c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/kube-rbac-proxy-rules/0.log"
Oct 30 23:43:55.300255 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.300227    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6776c6b778-2gtz5_c754a9ec-30eb-4e30-a6ca-f2f2f23fe87f/kube-rbac-proxy-metrics/0.log"
Oct 30 23:43:55.633416 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.633373    5503 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"]
Oct 30 23:43:55.633901 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.633880    5503 cpu_manager.go:414] "containerMap: removing container" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" containerName="main"
Oct 30 23:43:55.634008 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.633902    5503 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" containerName="main"
Oct 30 23:43:55.634065 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.634015    5503 memory_manager.go:365] "RemoveStaleState removing state" podUID="28093edf-3a2d-4441-a913-e809927b4cf2" containerName="main"
Oct 30 23:43:55.637254 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.637231    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.648212 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.648183    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dn4ld\"/\"kube-root-ca.crt\""
Oct 30 23:43:55.650076 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.650052    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"]
Oct 30 23:43:55.653944 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.653915    5503 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dn4ld\"/\"default-dockercfg-57457\""
Oct 30 23:43:55.664145 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.664117    5503 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dn4ld\"/\"openshift-service-ca.crt\""
Oct 30 23:43:55.703197 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.703158    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-podres\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.703414 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.703220    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-sys\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.703414 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.703267    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7fvs\" (UniqueName: \"kubernetes.io/projected/d5521c59-52c2-4efd-8fcc-67a9ca93923f-kube-api-access-j7fvs\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.703414 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.703396    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-proc\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.703548 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.703425    5503 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-lib-modules\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.804542 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.804498    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-sys\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.804542 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.804538    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7fvs\" (UniqueName: \"kubernetes.io/projected/d5521c59-52c2-4efd-8fcc-67a9ca93923f-kube-api-access-j7fvs\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.804820 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.804562    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-proc\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.804820 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.804578    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-lib-modules\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.804820 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.804625    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-sys\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.804820 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.804648    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-proc\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.804820 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.804764    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-lib-modules\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.804820 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.804773    5503 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-podres\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.805000 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.804856    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/d5521c59-52c2-4efd-8fcc-67a9ca93923f-podres\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.818506 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.818471    5503 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7fvs\" (UniqueName: \"kubernetes.io/projected/d5521c59-52c2-4efd-8fcc-67a9ca93923f-kube-api-access-j7fvs\") pod \"perf-node-gather-daemonset-ggvf5\" (UID: \"d5521c59-52c2-4efd-8fcc-67a9ca93923f\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:55.950556 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:55.950461    5503 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:56.235381 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:56.235352    5503 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"]
Oct 30 23:43:56.238173 ip-10-0-128-97 kubenswrapper[5503]: W1030 23:43:56.238141    5503 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd5521c59_52c2_4efd_8fcc_67a9ca93923f.slice/crio-368a308a448cd2c80162af7d376faf51030a31d5c4725d76b41afd29cc251c7b WatchSource:0}: Error finding container 368a308a448cd2c80162af7d376faf51030a31d5c4725d76b41afd29cc251c7b: Status 404 returned error can't find the container with id 368a308a448cd2c80162af7d376faf51030a31d5c4725d76b41afd29cc251c7b
Oct 30 23:43:56.833886 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:56.833847    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5" event={"ID":"d5521c59-52c2-4efd-8fcc-67a9ca93923f","Type":"ContainerStarted","Data":"9cfd987663a3da93900d2117e63cc3ce7521d6cfb9cbc42116a6d20a04be001a"}
Oct 30 23:43:56.834066 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:56.833904    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5" event={"ID":"d5521c59-52c2-4efd-8fcc-67a9ca93923f","Type":"ContainerStarted","Data":"368a308a448cd2c80162af7d376faf51030a31d5c4725d76b41afd29cc251c7b"}
Oct 30 23:43:56.834066 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:56.833949    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:43:56.854580 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:56.854520    5503 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5" podStartSLOduration=1.854506322 podStartE2EDuration="1.854506322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 23:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:43:56.853075126 +0000 UTC m=+5079.122156651" watchObservedRunningTime="2026-10-30 23:43:56.854506322 +0000 UTC m=+5079.123587906"
Oct 30 23:43:57.407737 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:57.407708    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6b88dcb6c8-v2bvm_4356691f-f269-47a4-84d8-c828d47d0b64/console/0.log"
Oct 30 23:43:59.198760 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:59.198717    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-96mpc_376fe364-ccc4-4435-b5e0-bef1e6715b26/dns/0.log"
Oct 30 23:43:59.226635 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:59.226601    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-96mpc_376fe364-ccc4-4435-b5e0-bef1e6715b26/kube-rbac-proxy/0.log"
Oct 30 23:43:59.402685 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:43:59.402639    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-wqsgd_0b6fa19a-6fb7-40b6-9073-0a52cfca6b79/dns-node-resolver/0.log"
Oct 30 23:44:00.045915 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:00.045882    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-dkmz4_84f3a1bb-9373-443e-b957-1bf6ccb03477/node-ca/0.log"
Oct 30 23:44:01.364347 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:01.364314    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-wr97s_40a5acae-35b7-4ac0-911b-7699d0127186/discovery/0.log"
Oct 30 23:44:01.404799 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:01.404761    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-2c7vh_62f56fdd-bb51-4c60-b597-323c0353b756/istio-proxy/0.log"
Oct 30 23:44:01.449468 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:01.449435    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-5cx6f_993bc0f1-0397-46de-ab78-8f014b5b39a4/istio-proxy/0.log"
Oct 30 23:44:01.678189 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:01.678112    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-t2dz9_339f08b1-7307-4020-8d26-50f702cb8148/istio-proxy/0.log"
Oct 30 23:44:01.717500 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:01.717465    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-wvd5p_37905f9c-9cf9-42e1-921f-33db04ec8ab9/istio-proxy/0.log"
Oct 30 23:44:02.580571 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:02.580539    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hrpz9_cd11fe2e-49c0-4478-928a-a04ed1f29273/serve-healthcheck-canary/0.log"
Oct 30 23:44:02.849710 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:02.849685    5503 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-ggvf5"
Oct 30 23:44:03.153014 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:03.152933    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5rttv_4e972ba1-2295-49d0-998b-00b86a1737af/kube-rbac-proxy/0.log"
Oct 30 23:44:03.181499 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:03.181466    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5rttv_4e972ba1-2295-49d0-998b-00b86a1737af/exporter/0.log"
Oct 30 23:44:03.215530 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:03.215497    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5rttv_4e972ba1-2295-49d0-998b-00b86a1737af/extractor/0.log"
Oct 30 23:44:07.389746 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:07.389712    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-76d8b77fbb-9nwtn_395cbd9e-c968-42ef-a3a4-ec39e0e6fceb/manager/0.log"
Oct 30 23:44:07.688106 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:07.687973    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-86c86ffb6b-w7p6x_d165075d-a6ec-456a-9fa3-25f176eef900/manager/0.log"
Oct 30 23:44:08.242321 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:08.242267    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-7f9f8cdc65-nq8qs_6718fe7a-ba3b-4e93-affc-3ebd5a036649/manager/0.log"
Oct 30 23:44:13.494784 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:13.494709    5503 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-cmh6h_f0f05f12-85f1-4c1d-9115-e1cd2a336b55/konnectivity-agent/0.log" limit=1024
Oct 30 23:44:15.278900 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:15.278861    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4gvnm_f6f0136b-88a1-427c-92bc-5cc7ed9d7030/kube-multus/0.log"
Oct 30 23:44:15.391607 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:44:15.391562    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:44:15.819607 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:15.819574    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kl9nq_5f9fcff5-40fb-422e-9319-50a1c4156131/kube-multus-additional-cni-plugins/0.log"
Oct 30 23:44:15.849701 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:15.849612    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kl9nq_5f9fcff5-40fb-422e-9319-50a1c4156131/egress-router-binary-copy/0.log"
Oct 30 23:44:15.892814 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:15.892757    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kl9nq_5f9fcff5-40fb-422e-9319-50a1c4156131/cni-plugins/0.log"
Oct 30 23:44:15.935792 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:15.935739    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kl9nq_5f9fcff5-40fb-422e-9319-50a1c4156131/bond-cni-plugin/0.log"
Oct 30 23:44:15.968355 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:15.968284    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kl9nq_5f9fcff5-40fb-422e-9319-50a1c4156131/routeoverride-cni/0.log"
Oct 30 23:44:15.996309 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:15.996253    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kl9nq_5f9fcff5-40fb-422e-9319-50a1c4156131/whereabouts-cni-bincopy/0.log"
Oct 30 23:44:16.044975 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:16.044942    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kl9nq_5f9fcff5-40fb-422e-9319-50a1c4156131/whereabouts-cni/0.log"
Oct 30 23:44:16.142769 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:16.142731    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4pvll_e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1/network-metrics-daemon/0.log"
Oct 30 23:44:16.234350 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:16.234318    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4pvll_e27e04a7-2065-4d2e-84fc-4dd5eeabb6c1/kube-rbac-proxy/0.log"
Oct 30 23:44:17.840697 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:17.840616    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82mjm_fb7545c5-92b9-4107-93d0-f0617277df0f/ovn-controller/0.log"
Oct 30 23:44:17.879418 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:17.879385    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82mjm_fb7545c5-92b9-4107-93d0-f0617277df0f/ovn-acl-logging/0.log"
Oct 30 23:44:17.918682 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:17.918647    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82mjm_fb7545c5-92b9-4107-93d0-f0617277df0f/kube-rbac-proxy-node/0.log"
Oct 30 23:44:17.961041 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:17.961014    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82mjm_fb7545c5-92b9-4107-93d0-f0617277df0f/kube-rbac-proxy-ovn-metrics/0.log"
Oct 30 23:44:17.993463 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:17.993433    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82mjm_fb7545c5-92b9-4107-93d0-f0617277df0f/northd/0.log"
Oct 30 23:44:18.034368 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:18.034318    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82mjm_fb7545c5-92b9-4107-93d0-f0617277df0f/nbdb/0.log"
Oct 30 23:44:18.069126 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:18.069097    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82mjm_fb7545c5-92b9-4107-93d0-f0617277df0f/sbdb/0.log"
Oct 30 23:44:18.155335 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:18.155285    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-82mjm_fb7545c5-92b9-4107-93d0-f0617277df0f/ovnkube-controller/0.log"
Oct 30 23:44:20.435867 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:20.435834    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-gtwvr_7cd8a3d9-fbbb-43c0-9920-345d2ab627c3/network-check-target-container/0.log"
Oct 30 23:44:22.007774 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:22.007739    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-5f2j9_d967b5ff-61dc-4c9a-8705-d735e0075db5/iptables-alerter/0.log"
Oct 30 23:44:23.174161 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:23.174127    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-7wqr7_21192d3f-1a3f-4a32-ab4a-70fbcada9424/tuned/0.log"
Oct 30 23:44:28.038388 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:28.038337    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-h8xmf_f3ef0c70-9090-44d0-bb59-95ab1cf0c245/csi-driver/0.log"
Oct 30 23:44:28.071196 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:28.071169    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-h8xmf_f3ef0c70-9090-44d0-bb59-95ab1cf0c245/csi-node-driver-registrar/0.log"
Oct 30 23:44:28.100646 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:28.100616    5503 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-h8xmf_f3ef0c70-9090-44d0-bb59-95ab1cf0c245/csi-liveness-probe/0.log"
Oct 30 23:44:42.494252 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:42.494212    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"]
Oct 30 23:44:42.494761 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:42.494416    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" podUID="62f56fdd-bb51-4c60-b597-323c0353b756" containerName="istio-proxy" containerID="cri-o://681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c" gracePeriod=30
Oct 30 23:44:42.499478 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:42.499029    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"]
Oct 30 23:44:42.499478 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:42.499282    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" podUID="993bc0f1-0397-46de-ab78-8f014b5b39a4" containerName="istio-proxy" containerID="cri-o://95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85" gracePeriod=30
Oct 30 23:44:42.500943 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:42.500915    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"]
Oct 30 23:44:42.501181 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:42.501158    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" podUID="37905f9c-9cf9-42e1-921f-33db04ec8ab9" containerName="istio-proxy" containerID="cri-o://3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155" gracePeriod=30
Oct 30 23:44:42.502905 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:42.502878    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"]
Oct 30 23:44:42.503181 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:42.503135    5503 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" podUID="339f08b1-7307-4020-8d26-50f702cb8148" containerName="istio-proxy" containerID="cri-o://7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05" gracePeriod=30
Oct 30 23:44:45.390401 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:44:45.390353    5503 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5rttv" containerName="extractor"
Oct 30 23:44:47.778859 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.778815    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:44:47.816802 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.816768    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:44:47.857451 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.857412    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:44:47.861403 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.861371    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:44:47.894369 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894328    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-socket\") pod \"993bc0f1-0397-46de-ab78-8f014b5b39a4\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") "
Oct 30 23:44:47.894555 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894415    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-podinfo\") pod \"993bc0f1-0397-46de-ab78-8f014b5b39a4\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") "
Oct 30 23:44:47.894555 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894499    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-token\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-token\") pod \"993bc0f1-0397-46de-ab78-8f014b5b39a4\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") "
Oct 30 23:44:47.894555 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894543    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-data\") pod \"993bc0f1-0397-46de-ab78-8f014b5b39a4\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") "
Oct 30 23:44:47.894738 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894577    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/993bc0f1-0397-46de-ab78-8f014b5b39a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993bc0f1-0397-46de-ab78-8f014b5b39a4-istiod-ca-cert\") pod \"993bc0f1-0397-46de-ab78-8f014b5b39a4\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") "
Oct 30 23:44:47.894738 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894619    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-kube-api-access-z57vx\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-kube-api-access-z57vx\") pod \"993bc0f1-0397-46de-ab78-8f014b5b39a4\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") "
Oct 30 23:44:47.894738 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894642    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-socket" pod "993bc0f1-0397-46de-ab78-8f014b5b39a4" (UID: "993bc0f1-0397-46de-ab78-8f014b5b39a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.894738 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894701    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-certs\") pod \"993bc0f1-0397-46de-ab78-8f014b5b39a4\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") "
Oct 30 23:44:47.894925 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894755    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-envoy\") pod \"993bc0f1-0397-46de-ab78-8f014b5b39a4\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") "
Oct 30 23:44:47.894925 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.894785    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-credential-socket\") pod \"993bc0f1-0397-46de-ab78-8f014b5b39a4\" (UID: \"993bc0f1-0397-46de-ab78-8f014b5b39a4\") "
Oct 30 23:44:47.895111 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.895092    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:47.895178 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.895092    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/993bc0f1-0397-46de-ab78-8f014b5b39a4-istiod-ca-cert" pod "993bc0f1-0397-46de-ab78-8f014b5b39a4" (UID: "993bc0f1-0397-46de-ab78-8f014b5b39a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:44:47.895392 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.895332    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-credential-socket" pod "993bc0f1-0397-46de-ab78-8f014b5b39a4" (UID: "993bc0f1-0397-46de-ab78-8f014b5b39a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.895483 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.895424    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-certs" pod "993bc0f1-0397-46de-ab78-8f014b5b39a4" (UID: "993bc0f1-0397-46de-ab78-8f014b5b39a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.895807 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.895785    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-data" pod "993bc0f1-0397-46de-ab78-8f014b5b39a4" (UID: "993bc0f1-0397-46de-ab78-8f014b5b39a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.897226 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.897186    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-podinfo" pod "993bc0f1-0397-46de-ab78-8f014b5b39a4" (UID: "993bc0f1-0397-46de-ab78-8f014b5b39a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:44:47.897629 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.897598    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-kube-api-access-z57vx" pod "993bc0f1-0397-46de-ab78-8f014b5b39a4" (UID: "993bc0f1-0397-46de-ab78-8f014b5b39a4"). InnerVolumeSpecName "kube-api-access-z57vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:47.897863 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.897842    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-token" pod "993bc0f1-0397-46de-ab78-8f014b5b39a4" (UID: "993bc0f1-0397-46de-ab78-8f014b5b39a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:47.898928 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.898904    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-envoy" pod "993bc0f1-0397-46de-ab78-8f014b5b39a4" (UID: "993bc0f1-0397-46de-ab78-8f014b5b39a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.995938 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.995905    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-data\") pod \"339f08b1-7307-4020-8d26-50f702cb8148\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") "
Oct 30 23:44:47.996149 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.995949    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-kube-api-access-gjtzn\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-kube-api-access-gjtzn\") pod \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") "
Oct 30 23:44:47.996149 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.995977    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-certs\") pod \"62f56fdd-bb51-4c60-b597-323c0353b756\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") "
Oct 30 23:44:47.996149 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.995993    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-envoy\") pod \"339f08b1-7307-4020-8d26-50f702cb8148\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") "
Oct 30 23:44:47.996149 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996018    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-istio-token\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-istio-token\") pod \"62f56fdd-bb51-4c60-b597-323c0353b756\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") "
Oct 30 23:44:47.996149 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996034    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-credential-socket\") pod \"62f56fdd-bb51-4c60-b597-323c0353b756\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") "
Oct 30 23:44:47.996149 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996070    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/62f56fdd-bb51-4c60-b597-323c0353b756-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f56fdd-bb51-4c60-b597-323c0353b756-istio-podinfo\") pod \"62f56fdd-bb51-4c60-b597-323c0353b756\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") "
Oct 30 23:44:47.996149 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996098    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-socket\") pod \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") "
Oct 30 23:44:47.996149 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996128    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/339f08b1-7307-4020-8d26-50f702cb8148-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/339f08b1-7307-4020-8d26-50f702cb8148-istiod-ca-cert\") pod \"339f08b1-7307-4020-8d26-50f702cb8148\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") "
Oct 30 23:44:47.996538 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996163    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istiod-ca-cert\") pod \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") "
Oct 30 23:44:47.996538 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996197    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-istio-token\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-istio-token\") pod \"339f08b1-7307-4020-8d26-50f702cb8148\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") "
Oct 30 23:44:47.996538 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996251    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-data" pod "339f08b1-7307-4020-8d26-50f702cb8148" (UID: "339f08b1-7307-4020-8d26-50f702cb8148"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.996538 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996341    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-certs" pod "62f56fdd-bb51-4c60-b597-323c0353b756" (UID: "62f56fdd-bb51-4c60-b597-323c0353b756"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.996538 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996433    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-credential-socket" pod "62f56fdd-bb51-4c60-b597-323c0353b756" (UID: "62f56fdd-bb51-4c60-b597-323c0353b756"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.996856 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996540    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-socket" pod "37905f9c-9cf9-42e1-921f-33db04ec8ab9" (UID: "37905f9c-9cf9-42e1-921f-33db04ec8ab9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.996856 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996690    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/339f08b1-7307-4020-8d26-50f702cb8148-istiod-ca-cert" pod "339f08b1-7307-4020-8d26-50f702cb8148" (UID: "339f08b1-7307-4020-8d26-50f702cb8148"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:44:47.996856 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996721    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-token\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-token\") pod \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") "
Oct 30 23:44:47.996995 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.996877    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istiod-ca-cert" pod "37905f9c-9cf9-42e1-921f-33db04ec8ab9" (UID: "37905f9c-9cf9-42e1-921f-33db04ec8ab9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:44:47.997079 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.997046    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-certs\") pod \"339f08b1-7307-4020-8d26-50f702cb8148\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") "
Oct 30 23:44:47.998930 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.997566    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-certs" pod "339f08b1-7307-4020-8d26-50f702cb8148" (UID: "339f08b1-7307-4020-8d26-50f702cb8148"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.998930 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.997708    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-certs\") pod \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") "
Oct 30 23:44:47.998930 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.997788    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-envoy\") pod \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") "
Oct 30 23:44:47.998930 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.997892    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-certs" pod "37905f9c-9cf9-42e1-921f-33db04ec8ab9" (UID: "37905f9c-9cf9-42e1-921f-33db04ec8ab9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.998930 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.998222    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-data\") pod \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") "
Oct 30 23:44:47.998930 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.998800    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-data" pod "37905f9c-9cf9-42e1-921f-33db04ec8ab9" (UID: "37905f9c-9cf9-42e1-921f-33db04ec8ab9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.999316 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999037    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f56fdd-bb51-4c60-b597-323c0353b756-istiod-ca-cert" pod "62f56fdd-bb51-4c60-b597-323c0353b756" (UID: "62f56fdd-bb51-4c60-b597-323c0353b756"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:44:47.999316 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999112    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62f56fdd-bb51-4c60-b597-323c0353b756-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f56fdd-bb51-4c60-b597-323c0353b756-istiod-ca-cert\") pod \"62f56fdd-bb51-4c60-b597-323c0353b756\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") "
Oct 30 23:44:47.999316 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999155    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-kube-api-access-tn7zb\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-kube-api-access-tn7zb\") pod \"62f56fdd-bb51-4c60-b597-323c0353b756\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") "
Oct 30 23:44:47.999316 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999222    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-kube-api-access-mqh8v\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-kube-api-access-mqh8v\") pod \"339f08b1-7307-4020-8d26-50f702cb8148\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") "
Oct 30 23:44:47.999316 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999263    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/339f08b1-7307-4020-8d26-50f702cb8148-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/339f08b1-7307-4020-8d26-50f702cb8148-istio-podinfo\") pod \"339f08b1-7307-4020-8d26-50f702cb8148\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") "
Oct 30 23:44:47.999316 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999290    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62f56fdd-bb51-4c60-b597-323c0353b756-istio-podinfo" pod "62f56fdd-bb51-4c60-b597-323c0353b756" (UID: "62f56fdd-bb51-4c60-b597-323c0353b756"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:44:47.999652 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999360    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-envoy" pod "339f08b1-7307-4020-8d26-50f702cb8148" (UID: "339f08b1-7307-4020-8d26-50f702cb8148"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.999652 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999629    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-istio-token" pod "339f08b1-7307-4020-8d26-50f702cb8148" (UID: "339f08b1-7307-4020-8d26-50f702cb8148"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:47.999790 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999731    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-socket\") pod \"339f08b1-7307-4020-8d26-50f702cb8148\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") "
Oct 30 23:44:47.999790 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999772    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-envoy\") pod \"62f56fdd-bb51-4c60-b597-323c0353b756\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") "
Oct 30 23:44:47.999900 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999819    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-podinfo\") pod \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") "
Oct 30 23:44:47.999900 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999858    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-data\") pod \"62f56fdd-bb51-4c60-b597-323c0353b756\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") "
Oct 30 23:44:47.999900 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999895    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-socket\") pod \"62f56fdd-bb51-4c60-b597-323c0353b756\" (UID: \"62f56fdd-bb51-4c60-b597-323c0353b756\") "
Oct 30 23:44:48.000046 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999937    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-credential-socket\") pod \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\" (UID: \"37905f9c-9cf9-42e1-921f-33db04ec8ab9\") "
Oct 30 23:44:48.000046 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:47.999972    5503 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-credential-socket\") pod \"339f08b1-7307-4020-8d26-50f702cb8148\" (UID: \"339f08b1-7307-4020-8d26-50f702cb8148\") "
Oct 30 23:44:48.000159 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.000090    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-kube-api-access-gjtzn" pod "37905f9c-9cf9-42e1-921f-33db04ec8ab9" (UID: "37905f9c-9cf9-42e1-921f-33db04ec8ab9"). InnerVolumeSpecName "kube-api-access-gjtzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:48.000558 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.000530    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-token" pod "37905f9c-9cf9-42e1-921f-33db04ec8ab9" (UID: "37905f9c-9cf9-42e1-921f-33db04ec8ab9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.000690    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-socket" pod "339f08b1-7307-4020-8d26-50f702cb8148" (UID: "339f08b1-7307-4020-8d26-50f702cb8148"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.000831    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-socket" pod "62f56fdd-bb51-4c60-b597-323c0353b756" (UID: "62f56fdd-bb51-4c60-b597-323c0353b756"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.000902    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-credential-socket" pod "37905f9c-9cf9-42e1-921f-33db04ec8ab9" (UID: "37905f9c-9cf9-42e1-921f-33db04ec8ab9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.000915    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-envoy" pod "37905f9c-9cf9-42e1-921f-33db04ec8ab9" (UID: "37905f9c-9cf9-42e1-921f-33db04ec8ab9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001113    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-credential-socket" pod "339f08b1-7307-4020-8d26-50f702cb8148" (UID: "339f08b1-7307-4020-8d26-50f702cb8148"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001287    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001305    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001314    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001323    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001331    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001405    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-data" pod "62f56fdd-bb51-4c60-b597-323c0353b756" (UID: "62f56fdd-bb51-4c60-b597-323c0353b756"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001547    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001573    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001589    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjtzn\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-kube-api-access-gjtzn\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.001600 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001609    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001624    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001638    5503 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-credential-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001684    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f56fdd-bb51-4c60-b597-323c0353b756-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001699    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001714    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001833    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/339f08b1-7307-4020-8d26-50f702cb8148-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001852    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001866    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001879    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001894    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001907    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001920    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001933    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001947    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/993bc0f1-0397-46de-ab78-8f014b5b39a4-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001967    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001981    5503 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f56fdd-bb51-4c60-b597-323c0353b756-istiod-ca-cert\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.001994    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z57vx\" (UniqueName: \"kubernetes.io/projected/993bc0f1-0397-46de-ab78-8f014b5b39a4-kube-api-access-z57vx\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.002009    5503 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/339f08b1-7307-4020-8d26-50f702cb8148-workload-socket\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.002413 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.002023    5503 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/993bc0f1-0397-46de-ab78-8f014b5b39a4-workload-certs\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.003360 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.002520    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-kube-api-access-tn7zb" pod "62f56fdd-bb51-4c60-b597-323c0353b756" (UID: "62f56fdd-bb51-4c60-b597-323c0353b756"). InnerVolumeSpecName "kube-api-access-tn7zb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:48.003360 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.002590    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/339f08b1-7307-4020-8d26-50f702cb8148-istio-podinfo" pod "339f08b1-7307-4020-8d26-50f702cb8148" (UID: "339f08b1-7307-4020-8d26-50f702cb8148"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:44:48.003360 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.002935    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-kube-api-access-mqh8v" pod "339f08b1-7307-4020-8d26-50f702cb8148" (UID: "339f08b1-7307-4020-8d26-50f702cb8148"). InnerVolumeSpecName "kube-api-access-mqh8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:48.003465 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.003400    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-envoy" pod "62f56fdd-bb51-4c60-b597-323c0353b756" (UID: "62f56fdd-bb51-4c60-b597-323c0353b756"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:48.003640 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.003608    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-istio-token" pod "62f56fdd-bb51-4c60-b597-323c0353b756" (UID: "62f56fdd-bb51-4c60-b597-323c0353b756"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:48.004817 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.004796    5503 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-podinfo" pod "37905f9c-9cf9-42e1-921f-33db04ec8ab9" (UID: "37905f9c-9cf9-42e1-921f-33db04ec8ab9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:44:48.089557 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.089454    5503 generic.go:356] "Generic (PLEG): container finished" podID="62f56fdd-bb51-4c60-b597-323c0353b756" containerID="681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c" exitCode=0
Oct 30 23:44:48.089557 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.089501    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" event={"ID":"62f56fdd-bb51-4c60-b597-323c0353b756","Type":"ContainerDied","Data":"681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c"}
Oct 30 23:44:48.089825 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.089565    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh" event={"ID":"62f56fdd-bb51-4c60-b597-323c0353b756","Type":"ContainerDied","Data":"a72b2870b45370fc64e3585cdc4aec4d0680bdb1e70dc8e255231b08aad2a7b5"}
Oct 30 23:44:48.089825 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.089511    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"
Oct 30 23:44:48.089825 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.089589    5503 scope.go:122] "RemoveContainer" containerID="681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c"
Oct 30 23:44:48.091249 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.091225    5503 generic.go:356] "Generic (PLEG): container finished" podID="37905f9c-9cf9-42e1-921f-33db04ec8ab9" containerID="3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155" exitCode=0
Oct 30 23:44:48.091357 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.091275    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"
Oct 30 23:44:48.091357 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.091305    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" event={"ID":"37905f9c-9cf9-42e1-921f-33db04ec8ab9","Type":"ContainerDied","Data":"3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155"}
Oct 30 23:44:48.091357 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.091349    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p" event={"ID":"37905f9c-9cf9-42e1-921f-33db04ec8ab9","Type":"ContainerDied","Data":"86aa8a303dc2e59a386647b9aa4fcc0a27696800dd3ccb7da4b108e9853a6947"}
Oct 30 23:44:48.092873 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.092849    5503 generic.go:356] "Generic (PLEG): container finished" podID="993bc0f1-0397-46de-ab78-8f014b5b39a4" containerID="95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85" exitCode=0
Oct 30 23:44:48.093003 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.092898    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"
Oct 30 23:44:48.093003 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.092925    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" event={"ID":"993bc0f1-0397-46de-ab78-8f014b5b39a4","Type":"ContainerDied","Data":"95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85"}
Oct 30 23:44:48.093003 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.092992    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f" event={"ID":"993bc0f1-0397-46de-ab78-8f014b5b39a4","Type":"ContainerDied","Data":"6b179edf6db10923391c3d72b822129354fda0484a350c11f9ce9d3c441ef144"}
Oct 30 23:44:48.097294 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.097271    5503 generic.go:356] "Generic (PLEG): container finished" podID="339f08b1-7307-4020-8d26-50f702cb8148" containerID="7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05" exitCode=0
Oct 30 23:44:48.097415 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.097315    5503 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"
Oct 30 23:44:48.097415 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.097322    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" event={"ID":"339f08b1-7307-4020-8d26-50f702cb8148","Type":"ContainerDied","Data":"7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05"}
Oct 30 23:44:48.097415 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.097361    5503 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9" event={"ID":"339f08b1-7307-4020-8d26-50f702cb8148","Type":"ContainerDied","Data":"3938b31bac519726d3401c0a74dcb30c1be95578fc8ebaf52673c4642174f009"}
Oct 30 23:44:48.101809 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.101788    5503 scope.go:122] "RemoveContainer" containerID="681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c"
Oct 30 23:44:48.102156 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:44:48.102129    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c\": container with ID starting with 681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c not found: ID does not exist" containerID="681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c"
Oct 30 23:44:48.102268 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.102168    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c"} err="failed to get container status \"681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c\": rpc error: code = NotFound desc = could not find container \"681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c\": container with ID starting with 681d03888b023c8649bf07e3e66409ce788fee4cb6c867ad08a4a20c17fd5a7c not found: ID does not exist"
Oct 30 23:44:48.102268 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.102191    5503 scope.go:122] "RemoveContainer" containerID="3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155"
Oct 30 23:44:48.102574 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.102554    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/37905f9c-9cf9-42e1-921f-33db04ec8ab9-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.102612 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.102581    5503 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-data\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.102612 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.102594    5503 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-istio-token\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.102612 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.102606    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tn7zb\" (UniqueName: \"kubernetes.io/projected/62f56fdd-bb51-4c60-b597-323c0353b756-kube-api-access-tn7zb\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.102781 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.102621    5503 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqh8v\" (UniqueName: \"kubernetes.io/projected/339f08b1-7307-4020-8d26-50f702cb8148-kube-api-access-mqh8v\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.102781 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.102634    5503 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/339f08b1-7307-4020-8d26-50f702cb8148-istio-podinfo\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.102781 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.102647    5503 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f56fdd-bb51-4c60-b597-323c0353b756-istio-envoy\") on node \"ip-10-0-128-97.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.113478 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.113454    5503 scope.go:122] "RemoveContainer" containerID="3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155"
Oct 30 23:44:48.113800 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:44:48.113781    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155\": container with ID starting with 3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155 not found: ID does not exist" containerID="3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155"
Oct 30 23:44:48.113863 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.113808    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155"} err="failed to get container status \"3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155\": rpc error: code = NotFound desc = could not find container \"3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155\": container with ID starting with 3f9c322f676c501003f928cd59115411edcf009bc493583050cf4e458133a155 not found: ID does not exist"
Oct 30 23:44:48.113863 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.113827    5503 scope.go:122] "RemoveContainer" containerID="95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85"
Oct 30 23:44:48.122650 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.122470    5503 scope.go:122] "RemoveContainer" containerID="95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85"
Oct 30 23:44:48.123075 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:44:48.122915    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85\": container with ID starting with 95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85 not found: ID does not exist" containerID="95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85"
Oct 30 23:44:48.123075 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.122977    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85"} err="failed to get container status \"95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85\": rpc error: code = NotFound desc = could not find container \"95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85\": container with ID starting with 95304cbe8d5555605069c2471dba6f8ce50f194cb8ceacdd82109a8575463c85 not found: ID does not exist"
Oct 30 23:44:48.123075 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.123003    5503 scope.go:122] "RemoveContainer" containerID="7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05"
Oct 30 23:44:48.125187 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.125162    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"]
Oct 30 23:44:48.130823 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.130799    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-2c7vh"]
Oct 30 23:44:48.132383 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.132364    5503 scope.go:122] "RemoveContainer" containerID="7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05"
Oct 30 23:44:48.132720 ip-10-0-128-97 kubenswrapper[5503]: E1030 23:44:48.132701    5503 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05\": container with ID starting with 7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05 not found: ID does not exist" containerID="7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05"
Oct 30 23:44:48.132771 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.132728    5503 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05"} err="failed to get container status \"7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05\": rpc error: code = NotFound desc = could not find container \"7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05\": container with ID starting with 7292750a807bde17baccd4a24eee007ef54f5eb8613f7451878d116947578e05 not found: ID does not exist"
Oct 30 23:44:48.143770 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.143742    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"]
Oct 30 23:44:48.149259 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.149235    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-wvd5p"]
Oct 30 23:44:48.167680 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.167641    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"]
Oct 30 23:44:48.173958 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.173930    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-5cx6f"]
Oct 30 23:44:48.194145 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.194116    5503 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"]
Oct 30 23:44:48.200609 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:48.200580    5503 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-t2dz9"]
Oct 30 23:44:49.854807 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:49.854769    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="339f08b1-7307-4020-8d26-50f702cb8148" path="/var/lib/kubelet/pods/339f08b1-7307-4020-8d26-50f702cb8148/volumes"
Oct 30 23:44:49.855291 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:49.855201    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37905f9c-9cf9-42e1-921f-33db04ec8ab9" path="/var/lib/kubelet/pods/37905f9c-9cf9-42e1-921f-33db04ec8ab9/volumes"
Oct 30 23:44:49.855580 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:49.855567    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62f56fdd-bb51-4c60-b597-323c0353b756" path="/var/lib/kubelet/pods/62f56fdd-bb51-4c60-b597-323c0353b756/volumes"
Oct 30 23:44:49.856328 ip-10-0-128-97 kubenswrapper[5503]: I1030 23:44:49.856310    5503 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="993bc0f1-0397-46de-ab78-8f014b5b39a4" path="/var/lib/kubelet/pods/993bc0f1-0397-46de-ab78-8f014b5b39a4/volumes"