Oct 30 22:18:17.819221 ip-10-0-132-192 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 22:18:18.146387 ip-10-0-132-192 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:18:18.146387 ip-10-0-132-192 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 22:18:18.146387 ip-10-0-132-192 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:18:18.146387 ip-10-0-132-192 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:18:18.151639 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151547    2569 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:18.151639 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151636    2569 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:18.151639 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151641    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:18.151639 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151645    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151650    2569 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151654    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151660    2569 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151664    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151668    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151671    2569 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151675    2569 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151678    2569 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151682    2569 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151685    2569 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151690    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151694    2569 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151698    2569 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151701    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151705    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151709    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151712    2569 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151717    2569 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151725    2569 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:18.151921 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151730    2569 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151734    2569 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151738    2569 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151742    2569 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151746    2569 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151750    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151754    2569 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151758    2569 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151762    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151766    2569 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151770    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151775    2569 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151779    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151783    2569 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151788    2569 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151792    2569 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151797    2569 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151801    2569 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151805    2569 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151809    2569 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:18.152693 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151813    2569 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151817    2569 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151821    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151826    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151830    2569 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151834    2569 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151838    2569 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151845    2569 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151852    2569 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151857    2569 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151864    2569 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151870    2569 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151876    2569 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151880    2569 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151884    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151888    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151892    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151916    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:18.153422 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151920    2569 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151925    2569 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151938    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151945    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151949    2569 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151954    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151957    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151962    2569 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151967    2569 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151972    2569 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151976    2569 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151980    2569 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151985    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151989    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151993    2569 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.151997    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152001    2569 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152006    2569 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152011    2569 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:18.153988 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152015    2569 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152019    2569 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152023    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152027    2569 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152031    2569 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152036    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152040    2569 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152045    2569 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152051    2569 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152056    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152060    2569 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152065    2569 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152069    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152073    2569 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152078    2569 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152083    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152087    2569 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152091    2569 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152095    2569 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152099    2569 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:18.154441 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152103    2569 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:18.155091 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152108    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:18.155091 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152113    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:18.155091 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152117    2569 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:18.155091 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152121    2569 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:18.155091 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.152127    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:18.155685 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155665    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:18.155685 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155685    2569 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:18.155685 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155690    2569 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155694    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155699    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155704    2569 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155708    2569 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155712    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155716    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155720    2569 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155724    2569 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155728    2569 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155732    2569 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155737    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155740    2569 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155744    2569 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155749    2569 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155753    2569 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155757    2569 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155760    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155764    2569 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:18.155866 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155768    2569 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155772    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155776    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155780    2569 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155784    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155788    2569 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155792    2569 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155797    2569 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155801    2569 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155806    2569 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155811    2569 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155815    2569 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155819    2569 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155823    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155828    2569 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155833    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155838    2569 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155842    2569 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155846    2569 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:18.156704 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155850    2569 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155854    2569 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155858    2569 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155863    2569 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155867    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155872    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155877    2569 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155882    2569 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155888    2569 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155917    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155924    2569 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155929    2569 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155933    2569 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155937    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155942    2569 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155946    2569 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155950    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155954    2569 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155959    2569 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:18.157542 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155962    2569 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155966    2569 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155971    2569 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155975    2569 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155980    2569 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155987    2569 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155993    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.155998    2569 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156003    2569 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156007    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156012    2569 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156016    2569 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156020    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156024    2569 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156028    2569 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156032    2569 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156037    2569 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156041    2569 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156045    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:18.158075 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156049    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156054    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156058    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156062    2569 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156066    2569 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156072    2569 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156076    2569 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156081    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156085    2569 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156090    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156094    2569 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156098    2569 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156102    2569 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156107    2569 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156111    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156116    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156120    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156124    2569 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156129    2569 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156133    2569 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:18.158556 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156139    2569 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156145    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156149    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156154    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156158    2569 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156163    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156167    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.156171    2569 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157033    2569 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157044    2569 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157049    2569 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157054    2569 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157058    2569 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157062    2569 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157067    2569 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157071    2569 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157075    2569 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157081    2569 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157086    2569 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157090    2569 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:18.159327 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157094    2569 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157098    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157103    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157107    2569 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157111    2569 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157115    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157120    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157125    2569 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157129    2569 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157134    2569 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157138    2569 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157142    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157146    2569 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157150    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157154    2569 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157159    2569 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157164    2569 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157168    2569 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157172    2569 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157176    2569 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:18.160213 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157180    2569 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157185    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157190    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157194    2569 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157198    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157202    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157206    2569 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157210    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157215    2569 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157220    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157224    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157229    2569 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157233    2569 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157238    2569 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157242    2569 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157254    2569 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157259    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157263    2569 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157267    2569 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:18.161111 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157271    2569 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157276    2569 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157282    2569 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157287    2569 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157291    2569 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157295    2569 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157300    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157304    2569 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157309    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157313    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157317    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157321    2569 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157325    2569 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157330    2569 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157335    2569 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157339    2569 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157343    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157347    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157352    2569 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157356    2569 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:18.161611 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157361    2569 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157365    2569 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157369    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157373    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157378    2569 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157382    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157389    2569 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157395    2569 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157399    2569 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157403    2569 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157407    2569 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157420    2569 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157424    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157428    2569 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157434    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157438    2569 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157445    2569 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157450    2569 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157454    2569 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:18.162286 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157458    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157462    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157467    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157471    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157475    2569 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157480    2569 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157484    2569 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157488    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157492    2569 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157496    2569 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157502    2569 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157506    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157511    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157515    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157519    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.157523    2569 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158765    2569 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158784    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158798    2569 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158806    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158813    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 22:18:18.162878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158818    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158824    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158831    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158837    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158842    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158848    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158853    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158858    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158863    2569 flags.go:64] FLAG: --cgroup-root=""
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158867    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158872    2569 flags.go:64] FLAG: --client-ca-file=""
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158876    2569 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158881    2569 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158888    2569 flags.go:64] FLAG: --cluster-domain=""
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158912    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158918    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158923    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158928    2569 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158934    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158939    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158944    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158949    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158954    2569 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158959    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 22:18:18.163387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158963    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158968    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158973    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158980    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158987    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158992    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.158998    2569 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159003    2569 flags.go:64] FLAG: --enable-server="true"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159008    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159014    2569 flags.go:64] FLAG: --event-burst="100"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159019    2569 flags.go:64] FLAG: --event-qps="50"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159024    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159029    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159034    2569 flags.go:64] FLAG: --eviction-hard=""
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159041    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159046    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159051    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159055    2569 flags.go:64] FLAG: --eviction-soft=""
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159060    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159065    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159070    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159075    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159079    2569 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159084    2569 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159088    2569 flags.go:64] FLAG: --feature-gates=""
Oct 30 22:18:18.164079 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159100    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159105    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159110    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159115    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159120    2569 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159125    2569 flags.go:64] FLAG: --help="false"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159130    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159135    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159140    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159145    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159150    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159159    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159164    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159168    2569 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159174    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159179    2569 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159184    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159189    2569 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159194    2569 flags.go:64] FLAG: --kube-reserved=""
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159199    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159203    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159208    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159213    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 22:18:18.164700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159218    2569 flags.go:64] FLAG: --lock-file=""
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159222    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159227    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159232    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159247    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159252    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159257    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159262    2569 flags.go:64] FLAG: --logging-format="text"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159267    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159273    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159278    2569 flags.go:64] FLAG: --manifest-url=""
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159282    2569 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159289    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159294    2569 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159301    2569 flags.go:64] FLAG: --max-pods="110"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159306    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159311    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159316    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159320    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159325    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159331    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159335    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159346    2569 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159351    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159357    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 22:18:18.165277 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159362    2569 flags.go:64] FLAG: --pod-cidr=""
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159367    2569 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159372    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159377    2569 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159381    2569 flags.go:64] FLAG: --port="10250"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159386    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159391    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0681f0f0ace93dac9"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159396    2569 flags.go:64] FLAG: --qos-reserved=""
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159401    2569 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159405    2569 flags.go:64] FLAG: --register-node="true"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159410    2569 flags.go:64] FLAG: --register-with-taints=""
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159417    2569 flags.go:64] FLAG: --registry-burst="10"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159422    2569 flags.go:64] FLAG: --registry-qps="5"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159426    2569 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159431    2569 flags.go:64] FLAG: --reserved-memory=""
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159437    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159441    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159446    2569 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159451    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159456    2569 flags.go:64] FLAG: --runonce="false"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159461    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159466    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159471    2569 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159476    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159482    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159488    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 22:18:18.165884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159492    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159499    2569 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159504    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159509    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159513    2569 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159518    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159524    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159529    2569 flags.go:64] FLAG: --system-cgroups=""
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159533    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159543    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159548    2569 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159552    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159560    2569 flags.go:64] FLAG: --tls-min-version=""
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159565    2569 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159569    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159574    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159579    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159583    2569 flags.go:64] FLAG: --v="2"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159590    2569 flags.go:64] FLAG: --version="false"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159597    2569 flags.go:64] FLAG: --vmodule=""
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159603    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.159608    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159855    2569 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159863    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:18.166582 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159868    2569 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159872    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159877    2569 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159881    2569 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159885    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159890    2569 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159918    2569 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159924    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159928    2569 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159934    2569 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159938    2569 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159942    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159946    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159951    2569 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159955    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159960    2569 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159965    2569 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159969    2569 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159973    2569 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159978    2569 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:18.167237 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159981    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159986    2569 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.159997    2569 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160001    2569 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160005    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160010    2569 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160014    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160019    2569 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160024    2569 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160028    2569 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160032    2569 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160037    2569 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160041    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160045    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160049    2569 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160053    2569 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160058    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160062    2569 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160068    2569 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160072    2569 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:18.167719 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160076    2569 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160082    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160088    2569 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160092    2569 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160096    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160101    2569 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160105    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160109    2569 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160114    2569 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160118    2569 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160122    2569 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160126    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160130    2569 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160134    2569 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160139    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160143    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160147    2569 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160151    2569 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160156    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:18.168210 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160160    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160167    2569 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160173    2569 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160178    2569 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160182    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160187    2569 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160191    2569 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160195    2569 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160200    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160204    2569 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160208    2569 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160216    2569 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160220    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160224    2569 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160228    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160232    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160236    2569 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160241    2569 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160245    2569 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:18.168658 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160250    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160254    2569 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160258    2569 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160263    2569 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160268    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160272    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160276    2569 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160280    2569 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160285    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160289    2569 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160293    2569 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160298    2569 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160302    2569 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160307    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160311    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160315    2569 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160319    2569 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160326    2569 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160332    2569 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:18.169126 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160337    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160342    2569 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160347    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160351    2569 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160355    2569 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160361    2569 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160366    2569 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.160415    2569 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160486    2569 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160495    2569 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160500    2569 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160506    2569 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160510    2569 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160515    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160519    2569 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160524    2569 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:18.169577 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160528    2569 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160533    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160537    2569 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160542    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160546    2569 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160551    2569 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160555    2569 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160559    2569 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160564    2569 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160568    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160572    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160576    2569 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160581    2569 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160585    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160589    2569 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160593    2569 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160597    2569 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160602    2569 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160606    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160610    2569 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:18.169978 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160615    2569 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160618    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160624    2569 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160628    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160633    2569 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160637    2569 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160641    2569 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160645    2569 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160649    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160653    2569 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160658    2569 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160662    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160666    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160670    2569 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160675    2569 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160680    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160684    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160690    2569 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160697    2569 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:18.170451 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160702    2569 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160706    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160711    2569 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160715    2569 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160719    2569 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160723    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160728    2569 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160732    2569 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160736    2569 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160741    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160745    2569 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160750    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160754    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160758    2569 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160762    2569 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160768    2569 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160774    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160778    2569 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160783    2569 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160787    2569 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:18.170890 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160791    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160795    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160800    2569 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160804    2569 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160808    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160812    2569 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160817    2569 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160821    2569 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160827    2569 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160831    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160835    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160839    2569 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160844    2569 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160850    2569 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160856    2569 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160860    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160864    2569 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160868    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160873    2569 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:18.171402 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160877    2569 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160881    2569 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160887    2569 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160891    2569 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160912    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160917    2569 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160921    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160925    2569 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160932    2569 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160937    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160941    2569 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160945    2569 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160952    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160957    2569 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160961    2569 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160965    2569 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160970    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160974    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160978    2569 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.160983    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:18.171882 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.169127    2569 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.169143    2569 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.169749    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.169756    2569 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169813    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169820    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169824    2569 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169826    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169829    2569 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169832    2569 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169834    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169837    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169840    2569 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169843    2569 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169846    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169848    2569 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169852    2569 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169854    2569 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169857    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:18.172368 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169859    2569 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169862    2569 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169864    2569 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169867    2569 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169870    2569 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169872    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169874    2569 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169877    2569 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169880    2569 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169882    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169885    2569 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169887    2569 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169890    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169908    2569 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169916    2569 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169919    2569 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169922    2569 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169924    2569 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169927    2569 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169929    2569 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:18.172799 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169932    2569 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169934    2569 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169936    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169939    2569 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169942    2569 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169944    2569 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169947    2569 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169949    2569 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169952    2569 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169954    2569 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169956    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169959    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169962    2569 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169964    2569 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169966    2569 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169969    2569 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169972    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169975    2569 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169977    2569 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169980    2569 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:18.173275 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169982    2569 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169985    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169988    2569 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169990    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169993    2569 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169996    2569 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.169998    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170001    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170004    2569 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170006    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170008    2569 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170011    2569 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170013    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170015    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170018    2569 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170022    2569 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170025    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170029    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170032    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:18.173750 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170034    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170038    2569 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170041    2569 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170044    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170047    2569 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170050    2569 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170052    2569 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170055    2569 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170057    2569 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170060    2569 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170063    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170065    2569 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170068    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170070    2569 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170073    2569 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170075    2569 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170078    2569 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170080    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170083    2569 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170086    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:18.174229 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170089    2569 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170092    2569 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170095    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170098    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170100    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170103    2569 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170105    2569 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170108    2569 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170110    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170113    2569 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170115    2569 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170118    2569 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.170146    2569 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170286    2569 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170290    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:18.174714 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170293    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170296    2569 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170298    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170301    2569 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170303    2569 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170307    2569 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170311    2569 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170314    2569 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170317    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170320    2569 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170323    2569 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170325    2569 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170328    2569 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170330    2569 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170333    2569 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170336    2569 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170339    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170341    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170344    2569 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170347    2569 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:18.175088 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170349    2569 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170352    2569 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170354    2569 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170358    2569 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170361    2569 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170363    2569 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170366    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170368    2569 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170371    2569 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170373    2569 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170376    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170378    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170381    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170383    2569 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170386    2569 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170388    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170391    2569 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170394    2569 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170396    2569 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:18.175550 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170398    2569 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170401    2569 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170403    2569 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170406    2569 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170408    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170411    2569 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170413    2569 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170415    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170419    2569 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170421    2569 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170424    2569 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170426    2569 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170429    2569 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170432    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170434    2569 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170437    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170439    2569 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170441    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:18.176020 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170444    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170446    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170449    2569 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170451    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170454    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170456    2569 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170459    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170462    2569 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170465    2569 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170468    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170470    2569 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170473    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170476    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170478    2569 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170480    2569 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170483    2569 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170485    2569 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170488    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170490    2569 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170492    2569 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:18.176437 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170495    2569 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170497    2569 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170500    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170503    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170505    2569 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170507    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170510    2569 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170512    2569 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170515    2569 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170518    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170520    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170523    2569 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170525    2569 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170528    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170531    2569 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170533    2569 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170535    2569 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170538    2569 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170540    2569 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170543    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:18.176914 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170545    2569 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170548    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170550    2569 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170552    2569 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170555    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170557    2569 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170559    2569 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.170581    2569 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170614    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170618    2569 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170621    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170623    2569 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170626    2569 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170629    2569 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170631    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170634    2569 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:18:18.177382 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170636    2569 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170639    2569 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170641    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170644    2569 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170647    2569 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170650    2569 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170652    2569 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170655    2569 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170657    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170659    2569 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170662    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170664    2569 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170667    2569 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170669    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170672    2569 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170675    2569 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170677    2569 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170680    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170682    2569 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:18:18.177763 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170685    2569 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170687    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170690    2569 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170692    2569 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170696    2569 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170700    2569 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170704    2569 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170707    2569 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170710    2569 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170713    2569 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170716    2569 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170718    2569 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170721    2569 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170723    2569 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170725    2569 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170728    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170730    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170732    2569 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170735    2569 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170738    2569 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:18:18.178336 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170741    2569 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170743    2569 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170746    2569 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170750    2569 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170753    2569 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170755    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170774    2569 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170778    2569 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170782    2569 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170784    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170787    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170790    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170792    2569 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170795    2569 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170797    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170799    2569 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170802    2569 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170805    2569 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170807    2569 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:18:18.179315 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170809    2569 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170812    2569 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170814    2569 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170817    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170820    2569 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170822    2569 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170824    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170827    2569 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170829    2569 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170832    2569 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170834    2569 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170836    2569 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170839    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170842    2569 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170844    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170847    2569 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170849    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170852    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170854    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:18:18.179760 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170856    2569 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170859    2569 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170861    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170864    2569 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170866    2569 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170869    2569 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170871    2569 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170874    2569 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170876    2569 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170878    2569 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170881    2569 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170884    2569 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170886    2569 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170888    2569 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170891    2569 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170906    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170908    2569 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170911    2569 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170914    2569 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170916    2569 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:18:18.180296 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:18.170919    2569 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:18:18.180849 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.170977    2569 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 22:18:18.180849 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.172988    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 22:18:18.180849 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.173760    2569 server.go:1014] "Starting client certificate rotation"
Oct 30 22:18:18.180849 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.173856    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 22:18:18.180849 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.173886    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 22:18:18.180849 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.175489    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:18:18.180849 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.180653    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:18:18.191490 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.191474    2569 log.go:25] "Validated CRI v1 runtime API"
Oct 30 22:18:18.196208 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.196190    2569 log.go:25] "Validated CRI v1 image API"
Oct 30 22:18:18.197089 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.197060    2569 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 22:18:18.200012 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.199992    2569 fs.go:135] Filesystem UUIDs: map[0903cb39-a7be-41be-93ed-3493de8db811:/dev/nvme0n1p3 1facfe9a-58f9-4bca-b5ae-2330f8940992:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 30 22:18:18.200074 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.200010    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 22:18:18.204937 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.204802    2569 manager.go:217] Machine: {Timestamp:2026-10-30 22:18:18.203605114 +0000 UTC m=+0.284191982 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099978 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e8f28d9cbbde52f989bb6ff709893 SystemUUID:ec2e8f28-d9cb-bde5-2f98-9bb6ff709893 BootID:5c1b667f-1918-45df-8038-4a4f26a8bcbd Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c3:18:3e:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c3:18:3e:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:62:48:84:9e:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 22:18:18.204937 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.204935    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 22:18:18.205058 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.205012    2569 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 22:18:18.207711 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.207678    2569 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 22:18:18.207868 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.207713    2569 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-192.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 22:18:18.207979 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.207872    2569 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 22:18:18.207979 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.207885    2569 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 22:18:18.207979 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.207970    2569 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:18:18.208087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.207995    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:18:18.208858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.208844    2569 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 22:18:18.209675 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.209658    2569 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 22:18:18.212497 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.212482    2569 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 22:18:18.212638 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.212627    2569 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 22:18:18.212693 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.212669    2569 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 22:18:18.212693 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.212683    2569 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 22:18:18.212774 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.212698    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 22:18:18.212774 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.212713    2569 kubelet.go:405] "Adding apiserver pod source"
Oct 30 22:18:18.212774 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.212725    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 22:18:18.213102 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.213088    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:18:18.214923 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.214883    2569 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 22:18:18.220072 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.220054    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:18:18.220792 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.220780    2569 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 22:18:18.220825 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.220811    2569 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 22:18:18.220989 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.220979    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 22:18:18.221034 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.220994    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 22:18:18.221034 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221000    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 22:18:18.221034 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221005    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 22:18:18.221034 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221011    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 22:18:18.221034 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221016    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 22:18:18.221540 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221531    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 22:18:18.221573 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221541    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 22:18:18.221573 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221548    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 22:18:18.221573 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221554    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 22:18:18.221573 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221563    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 22:18:18.221573 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.221572    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 22:18:18.222186 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.222177    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 22:18:18.222216 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.222187    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 22:18:18.225779 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.225766    2569 server.go:1263] "Started kubelet"
Oct 30 22:18:18.226178 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.226156    2569 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 22:18:18.226275 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.225869    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 22:18:18.226332 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.226305    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 22:18:18.226549 ip-10-0-132-192 systemd[1]: Started Kubernetes Kubelet.
Oct 30 22:18:18.227488 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.227392    2569 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 22:18:18.228007 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.227953    2569 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 22:18:18.229219 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.229198    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-192.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 22:18:18.232653 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.232631    2569 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 22:18:18.232887 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.232871    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 22:18:18.233436 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.233423    2569 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 22:18:18.233549 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.233516    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:18.234103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234091    2569 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 22:18:18.234356 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234104    2569 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 22:18:18.234356 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234221    2569 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 22:18:18.234356 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234228    2569 factory.go:55] Registering systemd factory
Oct 30 22:18:18.234356 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234242    2569 factory.go:223] Registration of the systemd container factory successfully
Oct 30 22:18:18.234356 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234229    2569 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 22:18:18.234700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234399    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 22:18:18.234700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234528    2569 factory.go:153] Registering CRI-O factory
Oct 30 22:18:18.234700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234539    2569 factory.go:223] Registration of the crio container factory successfully
Oct 30 22:18:18.234700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234588    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 22:18:18.234700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234605    2569 factory.go:103] Registering Raw factory
Oct 30 22:18:18.234700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.234618    2569 manager.go:1199] Started watching for new ooms in manager
Oct 30 22:18:18.235481 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.235463    2569 manager.go:319] Starting recovery of all containers
Oct 30 22:18:18.238061 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.238039    2569 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 22:18:18.238155 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.238120    2569 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-192.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 22:18:18.238332 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.238287    2569 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-192.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 22:18:18.239060 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.238171    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-192.us-east-2.compute.internal.18e36e9d3c64afae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-192.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-192.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 22:18:18.225741742 +0000 UTC m=+0.306328608,LastTimestamp:2026-10-30 22:18:18.225741742 +0000 UTC m=+0.306328608,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-192.us-east-2.compute.internal,}"
Oct 30 22:18:18.242284 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.242261    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hb9dn"
Oct 30 22:18:18.245511 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.245481    2569 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 22:18:18.248240 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.248221    2569 manager.go:324] Recovery completed
Oct 30 22:18:18.251907 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.251875    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hb9dn"
Oct 30 22:18:18.253246 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.253232    2569 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:18.254939 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.254926    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:18.255001 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.254960    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:18.255001 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.254970    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:18.255499 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.255486    2569 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:18.255573 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.255488    2569 cpu_manager.go:226] "Starting" policy="none"
Oct 30 22:18:18.255573 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.255512    2569 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 22:18:18.255573 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.255526    2569 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 22:18:18.256181 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.256166    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:18.256245 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.256195    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:18.256245 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.256210    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:18.257455 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.257357    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-192.us-east-2.compute.internal.18e36e9d3e223616  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-192.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-192.us-east-2.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-192.us-east-2.compute.internal,},FirstTimestamp:2026-10-30 22:18:18.25493967 +0000 UTC m=+0.335526536,LastTimestamp:2026-10-30 22:18:18.25493967 +0000 UTC m=+0.335526536,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-192.us-east-2.compute.internal,}"
Oct 30 22:18:18.257585 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.257574    2569 policy_none.go:50] "Start"
Oct 30 22:18:18.257624 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.257589    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 22:18:18.257624 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.257605    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 22:18:18.259666 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.259654    2569 policy_none.go:44] "Start"
Oct 30 22:18:18.297671 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.297472    2569 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 22:18:18.301615 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.301600    2569 manager.go:341] "Starting Device Plugin manager"
Oct 30 22:18:18.301666 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.301638    2569 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 22:18:18.301666 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.301656    2569 server.go:87] "Starting device plugin registration server"
Oct 30 22:18:18.301974 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.301963    2569 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 22:18:18.302005 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.301978    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 22:18:18.302128 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.302104    2569 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 22:18:18.302251 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.302202    2569 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 22:18:18.302251 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.302210    2569 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 22:18:18.302738 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.302708    2569 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 22:18:18.302815 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.302771    2569 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:18.402516 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.402452    2569 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:18.404351 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.403401    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:18.404351 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.403428    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:18.404351 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.403438    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:18.404351 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.403472    2569 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.411949 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.411928    2569 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.412033 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.411952    2569 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-192.us-east-2.compute.internal\": node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:18.426098 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.426078    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:18.489537 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.489486    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 22:18:18.490886 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.490869    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 22:18:18.490991 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.490892    2569 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 22:18:18.490991 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.490929    2569 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 22:18:18.491623 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.490960    2569 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 22:18:18.497859 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.497841    2569 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:18.527053 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.527012    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:18.592450 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.592409    2569 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"]
Oct 30 22:18:18.592578 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.592494    2569 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:18.593518 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.593504    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:18.593588 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.593532    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:18.593588 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.593543    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:18.595663 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.595651    2569 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:18.595807 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.595793    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.595846 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.595821    2569 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:18.596527 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.596510    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:18.596527 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.596522    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:18.596664 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.596543    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:18.596664 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.596546    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:18.596664 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.596558    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:18.596805 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.596560    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:18.598522 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.598505    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.598610 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.598533    2569 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:18:18.599237 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.599224    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:18:18.599306 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.599248    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:18:18.599306 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.599257    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:18:18.625401 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.625383    2569 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found" node="ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.627165 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.627118    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:18.628378 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.628361    2569 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found" node="ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.635390 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.635374    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/320724d28f4bb4eb7790f5cce736ad83-config\") pod \"kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"320724d28f4bb4eb7790f5cce736ad83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.635459 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.635399    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.635459 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.635417    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.727298 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.727223    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:18.735506 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.735484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/320724d28f4bb4eb7790f5cce736ad83-config\") pod \"kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"320724d28f4bb4eb7790f5cce736ad83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.735627 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.735512    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.735627 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.735532    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.735627 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.735583    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/320724d28f4bb4eb7790f5cce736ad83-config\") pod \"kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"320724d28f4bb4eb7790f5cce736ad83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.735627 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.735597    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.735771 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.735655    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.827685 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.827656    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:18.928177 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:18.928148    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:18.929267 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.929250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:18.931398 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:18.931383    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:19.028763 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:19.028684    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:19.129261 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:19.129221    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:19.174774 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.174748    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 22:18:19.175246 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.174885    2569 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 22:18:19.230237 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:19.230206    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:19.256523 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.256481    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:13:18 +0000 UTC" deadline="2028-04-14 22:07:49.333552561 +0000 UTC"
Oct 30 22:18:19.256523 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.256514    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12767h49m30.077042978s"
Oct 30 22:18:19.331277 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:19.331196    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:19.431955 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:19.431793    2569 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:18:19.452852 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.452826    2569 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:19.533801 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.533769    2569 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:19.546614 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.546551    2569 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:18:19.548076 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.548058    2569 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:18:19.555782 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.555760    2569 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:18:19.557054 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:19.557026    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222b3fa31d5bc6e9197fa2d0d2743eec.slice/crio-ddd09928a7a0cd62a016a44fcc552e3de824cf3a711ed70bddf0e9b82e669f35 WatchSource:0}: Error finding container ddd09928a7a0cd62a016a44fcc552e3de824cf3a711ed70bddf0e9b82e669f35: Status 404 returned error can't find the container with id ddd09928a7a0cd62a016a44fcc552e3de824cf3a711ed70bddf0e9b82e669f35
Oct 30 22:18:19.557721 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:19.557703    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320724d28f4bb4eb7790f5cce736ad83.slice/crio-13fbf1123d6b450a07f082ff67b0543ec582413e69ce9216baaaa4c443160063 WatchSource:0}: Error finding container 13fbf1123d6b450a07f082ff67b0543ec582413e69ce9216baaaa4c443160063: Status 404 returned error can't find the container with id 13fbf1123d6b450a07f082ff67b0543ec582413e69ce9216baaaa4c443160063
Oct 30 22:18:19.563312 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.563295    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:18:19.672352 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.672271    2569 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:19.793298 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:19.793264    2569 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:20.214000 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.213962    2569 apiserver.go:52] "Watching apiserver"
Oct 30 22:18:20.224753 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.224718    2569 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 22:18:20.225564 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.225535    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-cdmr4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk","openshift-cluster-node-tuning-operator/tuned-ht6mr","openshift-image-registry/node-ca-kx7l5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal","openshift-multus/multus-4bwqs","openshift-network-diagnostics/network-check-target-27nh7","openshift-network-operator/iptables-alerter-h6x2s","kube-system/global-pull-secret-syncer-bzxzm","kube-system/konnectivity-agent-ptpfs","openshift-dns/node-resolver-cf4tj","openshift-ovn-kubernetes/ovnkube-node-hdm75","kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal","openshift-multus/multus-additional-cni-plugins-gbckc"]
Oct 30 22:18:20.227994 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.227967    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:20.231226 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.231203    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.234745 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.232434    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 22:18:20.234745 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.232516    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 22:18:20.234745 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.232602    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jmj7l\""
Oct 30 22:18:20.234745 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.233746    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.234745 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.234201    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 22:18:20.235301 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.235256    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 22:18:20.235445 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.235311    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 22:18:20.235445 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.235438    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jb8sb\""
Oct 30 22:18:20.235546 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.235512    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 22:18:20.235676 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.235658    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.237733 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.237711    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 22:18:20.237831 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.237808    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:18:20.237958 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.237941    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tp94j\""
Oct 30 22:18:20.238041 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.238010    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:20.238041 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.238012    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 22:18:20.238169 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.238052    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:20.240083 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.240051    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.242372 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242353    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.242459 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.242420    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:20.242544 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242524    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:18:20.242653 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242638    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-systemd\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.242731 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242668    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-lib-modules\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.242784 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242747    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-host\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.242835 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242792    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-tmp\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.242887 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242844    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h824h\" (UniqueName: \"kubernetes.io/projected/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-kube-api-access-h824h\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.242954 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242889    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6695e08-7390-47f0-b5db-f474a466cf0a-agent-certs\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:20.242954 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242940    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlz2q\" (UniqueName: \"kubernetes.io/projected/777ef147-4f3f-4c89-ae2b-545aad6b31f1-kube-api-access-dlz2q\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.243053 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242956    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysconfig\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.243053 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242970    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.243053 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.242993    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-conf\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.243186 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243060    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-run\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.243186 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243116    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/777ef147-4f3f-4c89-ae2b-545aad6b31f1-serviceca\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.243186 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243139    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-kubernetes\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.243333 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243196    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-sys\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.243333 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243246    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-tuned\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.243333 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243276    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6695e08-7390-47f0-b5db-f474a466cf0a-konnectivity-ca\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:20.243333 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/777ef147-4f3f-4c89-ae2b-545aad6b31f1-host\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.243525 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243340    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xtbl\" (UniqueName: \"kubernetes.io/projected/bb6f24bb-ba37-400e-878f-267797c1903b-kube-api-access-6xtbl\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.243525 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243364    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-var-lib-kubelet\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.243525 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243387    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb6f24bb-ba37-400e-878f-267797c1903b-iptables-alerter-script\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.243525 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243409    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb6f24bb-ba37-400e-878f-267797c1903b-host-slash\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.243525 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.243431    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-modprobe-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.244435 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.244419    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.247849 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.246667    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.250691 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.249386    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:20.250691 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.249449    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:20.251481 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.251456    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.254105 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.254082    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.277896 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.277872    2569 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 22:18:20.278636 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.278126    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rmhxx"
Oct 30 22:18:20.290998 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.290969    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rmhxx"
Oct 30 22:18:20.332992 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.332958    2569 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:18:20.335414 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.335387    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 22:18:20.339453 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.339416    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 22:18:20.339682 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.339633    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 22:18:20.339782 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.339761    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qwcf4\""
Oct 30 22:18:20.339881 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.339838    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 22:18:20.339881 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.339857    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 22:18:20.339881 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.339859    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 22:18:20.339881 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.339870    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 22:18:20.340155 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.340107    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 22:18:20.340155 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.340139    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 22:18:20.340155 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.340139    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 22:18:20.340302 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.340171    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 22:18:20.340302 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.340197    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 22:18:20.341056 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341037    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-glgqx\""
Oct 30 22:18:20.341167 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341060    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-4x7f7\""
Oct 30 22:18:20.341167 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341083    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 22:18:20.341167 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341092    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 22:18:20.341167 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341140    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 22:18:20.341333 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341189    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 22:18:20.341333 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341262    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jm6gb\""
Oct 30 22:18:20.341524 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341506    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 22:18:20.341698 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341673    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-szxgd\""
Oct 30 22:18:20.341817 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341701    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 22:18:20.341817 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.341755    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kkwnr\""
Oct 30 22:18:20.342936 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.342915    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 22:18:20.344077 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344049    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-bin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.344175 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344148    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-conf-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.344235 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.344296 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.344368 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344302    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-k8s-cni-cncf-io\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.344421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344344    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-hostroot\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.344473 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-kubelet-config\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.344527 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb6f24bb-ba37-400e-878f-267797c1903b-host-slash\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.344583 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344547    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-systemd\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.344638 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344580    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb6f24bb-ba37-400e-878f-267797c1903b-host-slash\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.344638 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344628    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.344736 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344679    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.344736 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344712    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.344837 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-ovn\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.344837 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.344837 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344820    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cnibin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.345022 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344845    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-systemd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.345022 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-socket-dir-parent\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.345022 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344964    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jbvf\" (UniqueName: \"kubernetes.io/projected/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-kube-api-access-6jbvf\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.345022 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.344993    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/777ef147-4f3f-4c89-ae2b-545aad6b31f1-host\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.345022 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345010    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-etc-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.345278 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345076    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-systemd\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.345278 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345132    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-multus\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.345278 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345166    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-kubelet\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.345278 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345197    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/777ef147-4f3f-4c89-ae2b-545aad6b31f1-host\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.345278 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345243    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-system-cni-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.345278 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjdcx\" (UniqueName: \"kubernetes.io/projected/72a3a151-21d2-4551-94b5-332f0b4f9542-kube-api-access-cjdcx\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345314    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb6f24bb-ba37-400e-878f-267797c1903b-iptables-alerter-script\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345346    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-modprobe-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345373    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-tmp\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345379    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345391    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41e14309-abd7-407f-b98b-39005ec7d563-hosts-file\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345448    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-script-lib\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-netns\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345503    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-dbus\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345531    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.345571 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345589    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-system-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345651    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cni-binary-copy\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345659    2569 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345677    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345707    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-sys\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345733    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-sys-fs\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldx4p\" (UniqueName: \"kubernetes.io/projected/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kube-api-access-ldx4p\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345787    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-var-lib-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345815    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-binary-copy\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf9fr\" (UniqueName: \"kubernetes.io/projected/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-kube-api-access-lf9fr\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345868    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-socket-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345908    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-device-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.345974    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-slash\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346003    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-config\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.346087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346030    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-env-overrides\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346138    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76ab3954-a12f-484a-b32f-3de7517bc70f-ovn-node-metrics-cert\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346169    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h824h\" (UniqueName: \"kubernetes.io/projected/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-kube-api-access-h824h\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346195    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-netns\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346220    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346246    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6695e08-7390-47f0-b5db-f474a466cf0a-agent-certs\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlz2q\" (UniqueName: \"kubernetes.io/projected/777ef147-4f3f-4c89-ae2b-545aad6b31f1-kube-api-access-dlz2q\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-run\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346344    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-daemon-config\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346380    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-etc-kubernetes\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346406    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/777ef147-4f3f-4c89-ae2b-545aad6b31f1-serviceca\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-kubernetes\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346516    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqmvd\" (UniqueName: \"kubernetes.io/projected/41e14309-abd7-407f-b98b-39005ec7d563-kube-api-access-lqmvd\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346615    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-systemd-units\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346690    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-node-log\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.346732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346717    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346758    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6695e08-7390-47f0-b5db-f474a466cf0a-konnectivity-ca\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xtbl\" (UniqueName: \"kubernetes.io/projected/bb6f24bb-ba37-400e-878f-267797c1903b-kube-api-access-6xtbl\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-var-lib-kubelet\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346840    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41e14309-abd7-407f-b98b-39005ec7d563-tmp-dir\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346923    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-kubelet\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.346952    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9prx\" (UniqueName: \"kubernetes.io/projected/76ab3954-a12f-484a-b32f-3de7517bc70f-kube-api-access-l9prx\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347006    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-multus-certs\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347036    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-lib-modules\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-host\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347305    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-cnibin\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347330    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysconfig\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-conf\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347394    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-tuned\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347434    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-registration-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347476    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-log-socket\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.347547 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347488    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-conf\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.348404 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347509    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-bin\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.348404 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347537    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-netd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.348404 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.347674    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-modprobe-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.348404 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.348190    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-run\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.348404 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.348310    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb6f24bb-ba37-400e-878f-267797c1903b-iptables-alerter-script\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.348668 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.348410    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-sys\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.348668 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.348630    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/777ef147-4f3f-4c89-ae2b-545aad6b31f1-serviceca\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.348862 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.348841    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysconfig\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.349694 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.348961    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-lib-modules\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.349694 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.349034    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-host\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.349694 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.349057    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-kubernetes\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.349694 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.349067    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-var-lib-kubelet\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.349694 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.349285    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6695e08-7390-47f0-b5db-f474a466cf0a-konnectivity-ca\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:20.350715 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.350691    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-tmp\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.351519 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.351498    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6695e08-7390-47f0-b5db-f474a466cf0a-agent-certs\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:20.351824 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.351799    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-tuned\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.362064 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.362040    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h824h\" (UniqueName: \"kubernetes.io/projected/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-kube-api-access-h824h\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.362651 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.362621    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlz2q\" (UniqueName: \"kubernetes.io/projected/777ef147-4f3f-4c89-ae2b-545aad6b31f1-kube-api-access-dlz2q\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.363966 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.363944    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xtbl\" (UniqueName: \"kubernetes.io/projected/bb6f24bb-ba37-400e-878f-267797c1903b-kube-api-access-6xtbl\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.448369 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448317    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-etc-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448373    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-multus\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448402    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-kubelet\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448428    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-system-cni-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448436    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-etc-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448458    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjdcx\" (UniqueName: \"kubernetes.io/projected/72a3a151-21d2-4551-94b5-332f0b4f9542-kube-api-access-cjdcx\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448492    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-system-cni-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448510    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41e14309-abd7-407f-b98b-39005ec7d563-hosts-file\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448520    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-kubelet\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448530    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-multus\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.448560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448564    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-script-lib\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448591    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41e14309-abd7-407f-b98b-39005ec7d563-hosts-file\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448592    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-netns\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448639    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-netns\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448642    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-dbus\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448688    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448773    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-dbus\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448806    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448833    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-system-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448858    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cni-binary-copy\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448884    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448915    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-sys-fs\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448962    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldx4p\" (UniqueName: \"kubernetes.io/projected/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kube-api-access-ldx4p\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.448989    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-var-lib-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449016    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-binary-copy\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449044    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf9fr\" (UniqueName: \"kubernetes.io/projected/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-kube-api-access-lf9fr\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449071    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-socket-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.449103 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449098    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-device-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449155    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-slash\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449183    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-config\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449209    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-env-overrides\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449236    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76ab3954-a12f-484a-b32f-3de7517bc70f-ovn-node-metrics-cert\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449265    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-netns\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449293    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449323    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-daemon-config\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449348    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-etc-kubernetes\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449377    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449404    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqmvd\" (UniqueName: \"kubernetes.io/projected/41e14309-abd7-407f-b98b-39005ec7d563-kube-api-access-lqmvd\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-systemd-units\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449455    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-node-log\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449479    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449511    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41e14309-abd7-407f-b98b-39005ec7d563-tmp-dir\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449537    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-kubelet\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.449554    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449563    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9prx\" (UniqueName: \"kubernetes.io/projected/76ab3954-a12f-484a-b32f-3de7517bc70f-kube-api-access-l9prx\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.449918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449588    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-multus-certs\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.449612    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret podName:1a882ca4-1b11-494c-a51e-fb3c93ef5606 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:20.949595992 +0000 UTC m=+3.030182866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret") pod "global-pull-secret-syncer-bzxzm" (UID: "1a882ca4-1b11-494c-a51e-fb3c93ef5606") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449630    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-multus-certs\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-cnibin\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449779    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-registration-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-log-socket\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449837    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-bin\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449849    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-device-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449863    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-netd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.449876    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449890    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-bin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-conf-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449927    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-slash\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.449967    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs podName:7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa nodeName:}" failed. No retries permitted until 2026-10-30 22:18:20.949921515 +0000 UTC m=+3.030508456 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs") pod "network-metrics-daemon-cdmr4" (UID: "7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.449996    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-sys-fs\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.450797 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450029    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-system-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450037    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-conf-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450064    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450079    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-cnibin\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450080    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450120    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450127    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-log-socket\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-k8s-cni-cncf-io\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450161    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cni-binary-copy\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450171    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-bin\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450179    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-hostroot\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450209    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-kubelet-config\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450214    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-netd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450228    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-bin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450239    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450250    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450261    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-systemd-units\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.451633 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450268    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450269    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-ovn\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450315    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450335    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450378    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450419    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-node-log\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450455    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-ovn\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450550    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-var-lib-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451061    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-etc-kubernetes\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451108    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.450090    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-registration-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451304    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-config\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451385    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-k8s-cni-cncf-io\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451381    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-kubelet-config\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451427    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-kubelet\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451428    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-hostroot\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.452421 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451483    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451488    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41e14309-abd7-407f-b98b-39005ec7d563-tmp-dir\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cnibin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451546    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-systemd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451575    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-socket-dir-parent\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451590    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cnibin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451548    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-netns\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.452479    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-binary-copy\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451634    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-systemd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451705    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-socket-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451835    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451836    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-socket-dir-parent\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451913    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451956    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-env-overrides\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451966    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-script-lib\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.452438    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-daemon-config\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.453127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.451603    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jbvf\" (UniqueName: \"kubernetes.io/projected/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-kube-api-access-6jbvf\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.454858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.454830    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76ab3954-a12f-484a-b32f-3de7517bc70f-ovn-node-metrics-cert\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.462395 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.462342    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:20.462530 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.462401    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:20.462530 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.462447    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xrdzx for pod openshift-network-diagnostics/network-check-target-27nh7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:20.462530 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.462511    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx podName:3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:20.962491472 +0000 UTC m=+3.043078329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xrdzx" (UniqueName: "kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx") pod "network-check-target-27nh7" (UID: "3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:20.464191 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.464134    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqmvd\" (UniqueName: \"kubernetes.io/projected/41e14309-abd7-407f-b98b-39005ec7d563-kube-api-access-lqmvd\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.464807 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.464780    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjdcx\" (UniqueName: \"kubernetes.io/projected/72a3a151-21d2-4551-94b5-332f0b4f9542-kube-api-access-cjdcx\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.465293 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.465266    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldx4p\" (UniqueName: \"kubernetes.io/projected/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kube-api-access-ldx4p\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.465714 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.465693    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf9fr\" (UniqueName: \"kubernetes.io/projected/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-kube-api-access-lf9fr\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:20.466342 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.466313    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9prx\" (UniqueName: \"kubernetes.io/projected/76ab3954-a12f-484a-b32f-3de7517bc70f-kube-api-access-l9prx\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.467466 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.467442    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jbvf\" (UniqueName: \"kubernetes.io/projected/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-kube-api-access-6jbvf\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.497409 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.497343    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"320724d28f4bb4eb7790f5cce736ad83","Type":"ContainerStarted","Data":"13fbf1123d6b450a07f082ff67b0543ec582413e69ce9216baaaa4c443160063"}
Oct 30 22:18:20.498159 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.498127    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"222b3fa31d5bc6e9197fa2d0d2743eec","Type":"ContainerStarted","Data":"ddd09928a7a0cd62a016a44fcc552e3de824cf3a711ed70bddf0e9b82e669f35"}
Oct 30 22:18:20.546040 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.545998    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:20.553564 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.553530    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:18:20.561636 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:20.561581    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6695e08_7390_47f0_b5db_f474a466cf0a.slice/crio-5134b33f46911bea4675ef15db577a65f096367fe6edd4731de5e70512bf918e WatchSource:0}: Error finding container 5134b33f46911bea4675ef15db577a65f096367fe6edd4731de5e70512bf918e: Status 404 returned error can't find the container with id 5134b33f46911bea4675ef15db577a65f096367fe6edd4731de5e70512bf918e
Oct 30 22:18:20.565418 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:20.565392    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod777ef147_4f3f_4c89_ae2b_545aad6b31f1.slice/crio-caf23f7035b5d40672afc6969478cb1818857d9a1c0efb3528a36b85b488798a WatchSource:0}: Error finding container caf23f7035b5d40672afc6969478cb1818857d9a1c0efb3528a36b85b488798a: Status 404 returned error can't find the container with id caf23f7035b5d40672afc6969478cb1818857d9a1c0efb3528a36b85b488798a
Oct 30 22:18:20.565518 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.565482    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:18:20.574136 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.573961    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:18:20.578266 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:20.578237    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb6f24bb_ba37_400e_878f_267797c1903b.slice/crio-2d0e3b1f8b8dda90a7ff6b39e26ad3dcbbe6661ddbd61e1642bde37b87313fe6 WatchSource:0}: Error finding container 2d0e3b1f8b8dda90a7ff6b39e26ad3dcbbe6661ddbd61e1642bde37b87313fe6: Status 404 returned error can't find the container with id 2d0e3b1f8b8dda90a7ff6b39e26ad3dcbbe6661ddbd61e1642bde37b87313fe6
Oct 30 22:18:20.581472 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.581107    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4bwqs"
Oct 30 22:18:20.584371 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:20.584344    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc0d5ce3_964a_4b73_9a51_ee2b4c2aa654.slice/crio-9678f38ff04b55a78d7e10f408b1a8574280d43f9daa05e45bb9f8cc4fe3041a WatchSource:0}: Error finding container 9678f38ff04b55a78d7e10f408b1a8574280d43f9daa05e45bb9f8cc4fe3041a: Status 404 returned error can't find the container with id 9678f38ff04b55a78d7e10f408b1a8574280d43f9daa05e45bb9f8cc4fe3041a
Oct 30 22:18:20.592570 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:20.592548    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f66a94c_5332_47dc_b3bb_9d7ba8ebe1b6.slice/crio-c2f2693b5f0b2da6f487519177d18b9e4a12b66c5e724db5ee720653b48698c8 WatchSource:0}: Error finding container c2f2693b5f0b2da6f487519177d18b9e4a12b66c5e724db5ee720653b48698c8: Status 404 returned error can't find the container with id c2f2693b5f0b2da6f487519177d18b9e4a12b66c5e724db5ee720653b48698c8
Oct 30 22:18:20.610597 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.610568    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:18:20.612761 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.612732    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:18:20.616094 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.616074    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:18:20.632309 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:20.632284    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9093009_6c67_4d6b_8d3a_a388dbb078a3.slice/crio-592ebcb1e93b72d18af501402a503e32f6ec8bbf5c1019f4c15ba7b60040355b WatchSource:0}: Error finding container 592ebcb1e93b72d18af501402a503e32f6ec8bbf5c1019f4c15ba7b60040355b: Status 404 returned error can't find the container with id 592ebcb1e93b72d18af501402a503e32f6ec8bbf5c1019f4c15ba7b60040355b
Oct 30 22:18:20.634880 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.634681    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:20.646814 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:18:20.646771    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76ab3954_a12f_484a_b32f_3de7517bc70f.slice/crio-c134e4cc8deb953654e6b44f4087954e8815bd9e8a6aed22738757ff72a2f018 WatchSource:0}: Error finding container c134e4cc8deb953654e6b44f4087954e8815bd9e8a6aed22738757ff72a2f018: Status 404 returned error can't find the container with id c134e4cc8deb953654e6b44f4087954e8815bd9e8a6aed22738757ff72a2f018
Oct 30 22:18:20.956695 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.956604    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:20.956695 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:20.956659    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:20.956895 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.956803    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:20.956895 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.956869    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret podName:1a882ca4-1b11-494c-a51e-fb3c93ef5606 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:21.956850046 +0000 UTC m=+4.037436922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret") pod "global-pull-secret-syncer-bzxzm" (UID: "1a882ca4-1b11-494c-a51e-fb3c93ef5606") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:20.957367 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.957334    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:20.957466 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:20.957389    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs podName:7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa nodeName:}" failed. No retries permitted until 2026-10-30 22:18:21.95737428 +0000 UTC m=+4.037961137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs") pod "network-metrics-daemon-cdmr4" (UID: "7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:21.058498 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.057808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:21.058498 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.058010    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:21.058498 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.058034    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:21.058498 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.058050    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xrdzx for pod openshift-network-diagnostics/network-check-target-27nh7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:21.058498 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.058114    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx podName:3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:22.058094126 +0000 UTC m=+4.138681006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrdzx" (UniqueName: "kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx") pod "network-check-target-27nh7" (UID: "3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:21.294698 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.294514    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:13:20 +0000 UTC" deadline="2028-05-27 02:31:07.372561912 +0000 UTC"
Oct 30 22:18:21.294698 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.294590    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13780h12m46.077976009s"
Oct 30 22:18:21.492716 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.492052    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:21.492716 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.492204    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:21.492716 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.492288    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:21.492716 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.492366    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:21.507110 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.507074    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cf4tj" event={"ID":"41e14309-abd7-407f-b98b-39005ec7d563","Type":"ContainerStarted","Data":"af40adc363aca75eba397b156ee99c197cf1f27cfc50eb96a17acf47b6e1541a"}
Oct 30 22:18:21.520684 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.520594    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4bwqs" event={"ID":"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6","Type":"ContainerStarted","Data":"c2f2693b5f0b2da6f487519177d18b9e4a12b66c5e724db5ee720653b48698c8"}
Oct 30 22:18:21.535511 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.535466    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk" event={"ID":"d9093009-6c67-4d6b-8d3a-a388dbb078a3","Type":"ContainerStarted","Data":"592ebcb1e93b72d18af501402a503e32f6ec8bbf5c1019f4c15ba7b60040355b"}
Oct 30 22:18:21.538708 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.538673    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gbckc" event={"ID":"72a3a151-21d2-4551-94b5-332f0b4f9542","Type":"ContainerStarted","Data":"3e06b684166b1b72ff4fb79794ee5176b7133131d9d022fd8ebd1e287d7c2c59"}
Oct 30 22:18:21.540846 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.540816    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ht6mr" event={"ID":"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654","Type":"ContainerStarted","Data":"9678f38ff04b55a78d7e10f408b1a8574280d43f9daa05e45bb9f8cc4fe3041a"}
Oct 30 22:18:21.542952 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.542924    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h6x2s" event={"ID":"bb6f24bb-ba37-400e-878f-267797c1903b","Type":"ContainerStarted","Data":"2d0e3b1f8b8dda90a7ff6b39e26ad3dcbbe6661ddbd61e1642bde37b87313fe6"}
Oct 30 22:18:21.547010 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.546893    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kx7l5" event={"ID":"777ef147-4f3f-4c89-ae2b-545aad6b31f1","Type":"ContainerStarted","Data":"caf23f7035b5d40672afc6969478cb1818857d9a1c0efb3528a36b85b488798a"}
Oct 30 22:18:21.550580 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.550552    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ptpfs" event={"ID":"f6695e08-7390-47f0-b5db-f474a466cf0a","Type":"ContainerStarted","Data":"5134b33f46911bea4675ef15db577a65f096367fe6edd4731de5e70512bf918e"}
Oct 30 22:18:21.563460 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.563063    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" event={"ID":"76ab3954-a12f-484a-b32f-3de7517bc70f","Type":"ContainerStarted","Data":"c134e4cc8deb953654e6b44f4087954e8815bd9e8a6aed22738757ff72a2f018"}
Oct 30 22:18:21.968305 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.967291    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:21.968305 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:21.967541    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:21.968305 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.967739    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:21.968305 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.967803    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret podName:1a882ca4-1b11-494c-a51e-fb3c93ef5606 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:23.967783474 +0000 UTC m=+6.048370333 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret") pod "global-pull-secret-syncer-bzxzm" (UID: "1a882ca4-1b11-494c-a51e-fb3c93ef5606") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:21.968305 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.968220    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:21.968305 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:21.968268    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs podName:7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa nodeName:}" failed. No retries permitted until 2026-10-30 22:18:23.968253977 +0000 UTC m=+6.048840853 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs") pod "network-metrics-daemon-cdmr4" (UID: "7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:22.068119 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:22.068066    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:22.068299 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:22.068272    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:22.068392 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:22.068311    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:22.068392 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:22.068328    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xrdzx for pod openshift-network-diagnostics/network-check-target-27nh7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:22.068502 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:22.068393    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx podName:3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:24.068372242 +0000 UTC m=+6.148959114 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrdzx" (UniqueName: "kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx") pod "network-check-target-27nh7" (UID: "3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:22.295502 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:22.295367    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:13:20 +0000 UTC" deadline="2028-08-17 16:31:56.208737208 +0000 UTC"
Oct 30 22:18:22.295502 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:22.295420    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15762h13m33.913325478s"
Oct 30 22:18:22.494060 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:22.494024    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:22.494236 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:22.494173    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:22.583000 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:22.582890    2569 generic.go:356] "Generic (PLEG): container finished" podID="222b3fa31d5bc6e9197fa2d0d2743eec" containerID="c9c80e7efc26debb770d345768e655064766e7edd41fe8dd533e67fe52af05c7" exitCode=0
Oct 30 22:18:22.583174 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:22.583028    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"222b3fa31d5bc6e9197fa2d0d2743eec","Type":"ContainerDied","Data":"c9c80e7efc26debb770d345768e655064766e7edd41fe8dd533e67fe52af05c7"}
Oct 30 22:18:22.608770 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:22.608668    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"320724d28f4bb4eb7790f5cce736ad83","Type":"ContainerStarted","Data":"74f57e514d45c7b61590d6523f1c8059dfa300b42afcc5901c81f151ef90cf5e"}
Oct 30 22:18:22.627286 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:22.626285    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" podStartSLOduration=2.626266212 podStartE2EDuration="2.626266212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:22.625906219 +0000 UTC m=+4.706493099" watchObservedRunningTime="2026-10-30 22:18:22.626266212 +0000 UTC m=+4.706853093"
Oct 30 22:18:23.492263 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:23.491529    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:23.492263 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:23.491691    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:23.492263 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:23.492139    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:23.492263 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:23.492211    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:23.635747 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:23.635621    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"222b3fa31d5bc6e9197fa2d0d2743eec","Type":"ContainerStarted","Data":"0b45b9b038a49461f0288eaf4bbbeea28886fc94ab02ca1f7ce125b35ed9d57a"}
Oct 30 22:18:23.987682 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:23.987649    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:23.987682 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:23.987690    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:23.987854 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:23.987788    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:23.987854 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:23.987834    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret podName:1a882ca4-1b11-494c-a51e-fb3c93ef5606 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:27.987821095 +0000 UTC m=+10.068407953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret") pod "global-pull-secret-syncer-bzxzm" (UID: "1a882ca4-1b11-494c-a51e-fb3c93ef5606") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:23.988179 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:23.988164    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:23.988223 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:23.988206    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs podName:7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa nodeName:}" failed. No retries permitted until 2026-10-30 22:18:27.988194653 +0000 UTC m=+10.068781509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs") pod "network-metrics-daemon-cdmr4" (UID: "7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:24.088833 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:24.088785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:24.089021 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:24.088963    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:24.089021 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:24.088995    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:24.089021 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:24.089010    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xrdzx for pod openshift-network-diagnostics/network-check-target-27nh7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:24.089192 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:24.089075    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx podName:3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:28.089053771 +0000 UTC m=+10.169640629 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrdzx" (UniqueName: "kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx") pod "network-check-target-27nh7" (UID: "3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:24.492226 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:24.492150    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:24.492359 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:24.492304    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:24.643142 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:24.643100    2569 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="a2727832b69128ea52d11ae8a387dc30a8eb158558bf0ae43d3e3715337dd6e9" exitCode=0
Oct 30 22:18:24.644398 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:24.644366    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gbckc" event={"ID":"72a3a151-21d2-4551-94b5-332f0b4f9542","Type":"ContainerDied","Data":"a2727832b69128ea52d11ae8a387dc30a8eb158558bf0ae43d3e3715337dd6e9"}
Oct 30 22:18:24.673713 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:24.673645    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal" podStartSLOduration=4.673624488 podStartE2EDuration="4.673624488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:23.656478352 +0000 UTC m=+5.737065233" watchObservedRunningTime="2026-10-30 22:18:24.673624488 +0000 UTC m=+6.754211368"
Oct 30 22:18:25.491814 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:25.491780    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:25.492043 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:25.491938    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:25.492390 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:25.492368    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:25.492812 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:25.492479    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:25.654575 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:25.654516    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ptpfs" event={"ID":"f6695e08-7390-47f0-b5db-f474a466cf0a","Type":"ContainerStarted","Data":"f5762faaff88da9028de2b5b74db152801d1c96b4c04811f20e30eeb3d5e52a0"}
Oct 30 22:18:26.493386 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:26.492726    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:26.493386 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:26.492881    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:26.663072 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:26.663007    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kx7l5" event={"ID":"777ef147-4f3f-4c89-ae2b-545aad6b31f1","Type":"ContainerStarted","Data":"b25bf940377de7d81117b8ceef23f9311bae487fc3de9cbeb8fb1b13633a010e"}
Oct 30 22:18:26.687625 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:26.687555    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kx7l5" podStartSLOduration=3.3768736329999998 podStartE2EDuration="8.687535291s" totalImagesPullingTime="5.310661658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:26.68695512 +0000 UTC m=+8.767542001" watchObservedRunningTime="2026-10-30 22:18:26.687535291 +0000 UTC m=+8.768122170"
Oct 30 22:18:26.688018 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:26.687963    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ptpfs" podStartSLOduration=4.5870246340000005 podStartE2EDuration="8.687953643s" totalImagesPullingTime="4.100929009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:25.675264449 +0000 UTC m=+7.755851327" watchObservedRunningTime="2026-10-30 22:18:26.687953643 +0000 UTC m=+8.768540524"
Oct 30 22:18:27.492195 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:27.492097    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:27.492380 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:27.492257    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:27.492771 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:27.492753    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:27.492877 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:27.492852    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:27.669816 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:27.669773    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk" event={"ID":"d9093009-6c67-4d6b-8d3a-a388dbb078a3","Type":"ContainerStarted","Data":"6e68afd580b0715a0cef1237dbcdd7c4cb36872334bf43b36a300147b7da319f"}
Oct 30 22:18:28.027968 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:28.027147    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:28.027968 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:28.027202    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:28.027968 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:28.027359    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:28.027968 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:28.027430    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret podName:1a882ca4-1b11-494c-a51e-fb3c93ef5606 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:36.027410355 +0000 UTC m=+18.107997213 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret") pod "global-pull-secret-syncer-bzxzm" (UID: "1a882ca4-1b11-494c-a51e-fb3c93ef5606") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:28.027968 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:28.027851    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:28.027968 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:28.027919    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs podName:7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa nodeName:}" failed. No retries permitted until 2026-10-30 22:18:36.027888027 +0000 UTC m=+18.108474892 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs") pod "network-metrics-daemon-cdmr4" (UID: "7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:28.128146 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:28.128108    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:28.128317 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:28.128275    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:28.128317 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:28.128299    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:28.128317 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:28.128314    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xrdzx for pod openshift-network-diagnostics/network-check-target-27nh7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:28.128459 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:28.128375    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx podName:3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:36.128355377 +0000 UTC m=+18.208942248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrdzx" (UniqueName: "kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx") pod "network-check-target-27nh7" (UID: "3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:28.493989 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:28.493914    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:28.494149 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:28.494034    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:28.982408 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:28.982379    2569 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 22:18:29.214570 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.214362    2569 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:29.215357 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.215327    2569 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:29.313360 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.312544    2569 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:18:28.982401995Z","UUID":"91fd4992-24ed-4b0a-879c-1b2ecbc4c6c4","Handler":null,"Name":"","Endpoint":""}
Oct 30 22:18:29.314944 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.314441    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 22:18:29.314944 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.314467    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 22:18:29.492341 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.492126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:29.492469 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:29.492442    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:29.492708 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.492161    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:29.492708 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:29.492666    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:29.676526 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.676443    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cf4tj" event={"ID":"41e14309-abd7-407f-b98b-39005ec7d563","Type":"ContainerStarted","Data":"0abf996f51eb35565b2653a053df63506ccf3cb32050974d0b15df679feb039e"}
Oct 30 22:18:29.679181 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.679149    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4bwqs" event={"ID":"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6","Type":"ContainerStarted","Data":"3612016eb0c228e2aee0a874c414a43d413a6d570a63e2a67c57005be5957ca5"}
Oct 30 22:18:29.681696 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.681642    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk" event={"ID":"d9093009-6c67-4d6b-8d3a-a388dbb078a3","Type":"ContainerStarted","Data":"af47df543a025731312813da739b53533de30b0f7ee7a8ca462cd7f8fb5a1bbe"}
Oct 30 22:18:29.684104 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.684063    2569 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="720ff1050746f0fe69f31839fd57240b95afa88d4f756f28f027e3cee5c313c8" exitCode=0
Oct 30 22:18:29.684878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.684848    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gbckc" event={"ID":"72a3a151-21d2-4551-94b5-332f0b4f9542","Type":"ContainerDied","Data":"720ff1050746f0fe69f31839fd57240b95afa88d4f756f28f027e3cee5c313c8"}
Oct 30 22:18:29.717318 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.716588    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-cf4tj" podStartSLOduration=2.12490703 podStartE2EDuration="10.716569555s" totalImagesPullingTime="8.591662525s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:29.69597292 +0000 UTC m=+11.776559798" watchObservedRunningTime="2026-10-30 22:18:29.716569555 +0000 UTC m=+11.797156436"
Oct 30 22:18:29.717318 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:29.717103    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-4bwqs" podStartSLOduration=2.305962111 podStartE2EDuration="10.717091635s" totalImagesPullingTime="8.411129524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:29.716136536 +0000 UTC m=+11.796723413" watchObservedRunningTime="2026-10-30 22:18:29.717091635 +0000 UTC m=+11.797678552"
Oct 30 22:18:30.493707 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:30.493286    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:30.493707 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:30.493397    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:30.689296 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:30.689213    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk" event={"ID":"d9093009-6c67-4d6b-8d3a-a388dbb078a3","Type":"ContainerStarted","Data":"e9dcef9696fe764812e61a2ad427788b996ff91e9d9e851abc4f69c0c2166711"}
Oct 30 22:18:30.693372 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:30.693342    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gbckc" event={"ID":"72a3a151-21d2-4551-94b5-332f0b4f9542","Type":"ContainerStarted","Data":"91fd29068dc3c84e90b8ab22c147cb4018a0c572683a3758fc381c68b83d589c"}
Oct 30 22:18:30.696343 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:30.696316    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-h6x2s" event={"ID":"bb6f24bb-ba37-400e-878f-267797c1903b","Type":"ContainerStarted","Data":"c11cf4e8c4a6d08d91398cde4477b0b621b1bc5aacf25565cec775031f2dcc69"}
Oct 30 22:18:30.713852 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:30.713811    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk" podStartSLOduration=2.81832296 podStartE2EDuration="11.71379994s" totalImagesPullingTime="8.89547698s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:30.713259244 +0000 UTC m=+12.793846120" watchObservedRunningTime="2026-10-30 22:18:30.71379994 +0000 UTC m=+12.794386815"
Oct 30 22:18:30.756941 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:30.756875    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-h6x2s" podStartSLOduration=3.346539999 podStartE2EDuration="11.756856138s" totalImagesPullingTime="8.410316139s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:30.729716541 +0000 UTC m=+12.810303415" watchObservedRunningTime="2026-10-30 22:18:30.756856138 +0000 UTC m=+12.837443012"
Oct 30 22:18:31.491314 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:31.491279    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:31.491442 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:31.491326    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:31.491442 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:31.491427    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:31.491578 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:31.491552    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:31.701719 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:31.700572    2569 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="91fd29068dc3c84e90b8ab22c147cb4018a0c572683a3758fc381c68b83d589c" exitCode=0
Oct 30 22:18:31.701719 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:31.700656    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gbckc" event={"ID":"72a3a151-21d2-4551-94b5-332f0b4f9542","Type":"ContainerDied","Data":"91fd29068dc3c84e90b8ab22c147cb4018a0c572683a3758fc381c68b83d589c"}
Oct 30 22:18:31.703956 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:31.703924    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ht6mr" event={"ID":"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654","Type":"ContainerStarted","Data":"0debf146f7f63237e3acda06021390af87d29033ca7e53cb07f7c1a2b8c4d0bf"}
Oct 30 22:18:31.749843 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:31.749745    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ht6mr" podStartSLOduration=1.864165613 podStartE2EDuration="12.749731272s" totalImagesPullingTime="10.885565659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:31.749609309 +0000 UTC m=+13.830196184" watchObservedRunningTime="2026-10-30 22:18:31.749731272 +0000 UTC m=+13.830318146"
Oct 30 22:18:32.492268 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:32.492233    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:32.492454 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:32.492359    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:33.491291 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:33.491258    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:33.491291 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:33.491299    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:33.491835 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:33.491395    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:33.491835 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:33.491532    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:33.711957 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:33.711923    2569 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="ef1357a6657bca3759f69e45fee96d9d887c0ef97ce2a78d5bc3f1c168545725" exitCode=0
Oct 30 22:18:33.712101 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:33.711983    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gbckc" event={"ID":"72a3a151-21d2-4551-94b5-332f0b4f9542","Type":"ContainerDied","Data":"ef1357a6657bca3759f69e45fee96d9d887c0ef97ce2a78d5bc3f1c168545725"}
Oct 30 22:18:34.491446 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:34.491242    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:34.491939 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:34.491548    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:35.492113 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:35.492073    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:35.492601 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:35.492080    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:35.492601 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:35.492214    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:35.492601 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:35.492287    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:35.768874 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:35.768808    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cf4tj_41e14309-abd7-407f-b98b-39005ec7d563/dns-node-resolver/0.log"
Oct 30 22:18:36.089336 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.089174    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:36.089429 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.089358    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:36.089429 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:36.089315    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:36.089534 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:36.089455    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs podName:7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa nodeName:}" failed. No retries permitted until 2026-10-30 22:18:52.089439707 +0000 UTC m=+34.170026573 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs") pod "network-metrics-daemon-cdmr4" (UID: "7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:36.089534 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:36.089497    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:36.089620 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:36.089556    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret podName:1a882ca4-1b11-494c-a51e-fb3c93ef5606 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:52.089540304 +0000 UTC m=+34.170127159 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret") pod "global-pull-secret-syncer-bzxzm" (UID: "1a882ca4-1b11-494c-a51e-fb3c93ef5606") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:36.190140 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.190106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:36.190280 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:36.190263    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:36.190321 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:36.190285    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:36.190321 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:36.190295    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xrdzx for pod openshift-network-diagnostics/network-check-target-27nh7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:36.190382 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:36.190346    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx podName:3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04 nodeName:}" failed. No retries permitted until 2026-10-30 22:18:52.190331567 +0000 UTC m=+34.270918423 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrdzx" (UniqueName: "kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx") pod "network-check-target-27nh7" (UID: "3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:36.494674 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.494604    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:36.495051 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:36.494696    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:36.721880 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.721845    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" event={"ID":"76ab3954-a12f-484a-b32f-3de7517bc70f","Type":"ContainerStarted","Data":"d0ed85ae4ba58b147fc1b74a98e1d867f83a9434b49c42c624421833ee7a41cb"}
Oct 30 22:18:36.722036 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.721940    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" event={"ID":"76ab3954-a12f-484a-b32f-3de7517bc70f","Type":"ContainerStarted","Data":"88ec4e74397753249a4b4da2eb57e40ff3f1fc9a3499f36b7d8c5ae31ab85e84"}
Oct 30 22:18:36.722036 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.721972    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" event={"ID":"76ab3954-a12f-484a-b32f-3de7517bc70f","Type":"ContainerStarted","Data":"7585512acc8bc2ce44f143d57a604009960f8444790461f649aaa792abf602f1"}
Oct 30 22:18:36.722036 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.722003    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" event={"ID":"76ab3954-a12f-484a-b32f-3de7517bc70f","Type":"ContainerStarted","Data":"753ccc3dbc8a34a3afbaf946a7bb73bdba91533e2e1b8d45a8b6b64ebb30bc2a"}
Oct 30 22:18:36.722036 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.722035    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" event={"ID":"76ab3954-a12f-484a-b32f-3de7517bc70f","Type":"ContainerStarted","Data":"24b35c1cf195f1e885519ffdb00890c092ada07b81b966ab2aa34f1c674ebb4a"}
Oct 30 22:18:36.722215 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.722067    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" event={"ID":"76ab3954-a12f-484a-b32f-3de7517bc70f","Type":"ContainerStarted","Data":"2ee191ad8a7af6a676c66c35173b1be21803385d41aafdafd924c12620ff75e0"}
Oct 30 22:18:36.724744 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.724718    2569 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="5328723b5757d58ab13c91a0ce6391477ca1bcf62c642a4a79ecdb5573581fc4" exitCode=0
Oct 30 22:18:36.724880 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:36.724762    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gbckc" event={"ID":"72a3a151-21d2-4551-94b5-332f0b4f9542","Type":"ContainerDied","Data":"5328723b5757d58ab13c91a0ce6391477ca1bcf62c642a4a79ecdb5573581fc4"}
Oct 30 22:18:37.147570 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:37.147501    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kx7l5_777ef147-4f3f-4c89-ae2b-545aad6b31f1/node-ca/0.log"
Oct 30 22:18:37.492159 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:37.492066    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:37.492159 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:37.492094    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:37.492358 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:37.492170    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:37.492358 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:37.492312    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:37.729888 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:37.729855    2569 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="d9a75a9e7aaabbf10d9551700d00f04223522c9474d54bfe9d1ab05c80a4639f" exitCode=0
Oct 30 22:18:37.729888 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:37.729890    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gbckc" event={"ID":"72a3a151-21d2-4551-94b5-332f0b4f9542","Type":"ContainerDied","Data":"d9a75a9e7aaabbf10d9551700d00f04223522c9474d54bfe9d1ab05c80a4639f"}
Oct 30 22:18:38.492006 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:38.491972    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:38.492151 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:38.492055    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:38.735881 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:38.735839    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" event={"ID":"76ab3954-a12f-484a-b32f-3de7517bc70f","Type":"ContainerStarted","Data":"4fc020b5252e2a810a94c88e9cf5abd2c05732e679cb843f43940566895bab9f"}
Oct 30 22:18:38.739037 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:38.739004    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gbckc" event={"ID":"72a3a151-21d2-4551-94b5-332f0b4f9542","Type":"ContainerStarted","Data":"63aa45d587a4901b4f7e60925d7f1a8fec41a03c1b2f60ab697014acbdb89cf8"}
Oct 30 22:18:38.771831 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:38.771790    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gbckc" podStartSLOduration=8.355275218 podStartE2EDuration="19.771777765s" totalImagesPullingTime="11.416502547s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:38.771393994 +0000 UTC m=+20.851980869" watchObservedRunningTime="2026-10-30 22:18:38.771777765 +0000 UTC m=+20.852364636"
Oct 30 22:18:39.491668 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:39.491634    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:39.491668 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:39.491666    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:39.491855 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:39.491731    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:39.491893 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:39.491848    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:40.492159 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:40.492123    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:40.492764 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:40.492232    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:40.747042 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:40.746937    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" event={"ID":"76ab3954-a12f-484a-b32f-3de7517bc70f","Type":"ContainerStarted","Data":"bb455d044b5e6165e0c40927c938e194dba83400ff2ce99b2988c2d5a651ab91"}
Oct 30 22:18:40.747713 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:40.747466    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:40.747713 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:40.747648    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:40.765072 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:40.765047    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:40.765279 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:40.765262    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:40.784440 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:40.784386    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" podStartSLOduration=6.568744304 podStartE2EDuration="21.784371409s" totalImagesPullingTime="15.215627105s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:18:40.782692108 +0000 UTC m=+22.863278985" watchObservedRunningTime="2026-10-30 22:18:40.784371409 +0000 UTC m=+22.864958282"
Oct 30 22:18:41.492194 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:41.492023    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:41.492644 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:41.492097    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:41.492644 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:41.492276    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:41.492644 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:41.492348    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:41.749112 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:41.749036    2569 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:18:42.491715 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:42.491676    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:42.491938 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:42.491785    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:42.751650 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:42.751580    2569 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:18:43.158047 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:43.158017    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:43.158220 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:43.158139    2569 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:18:43.158665 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:43.158647    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:18:43.492088 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:43.492005    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:43.492239 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:43.492005    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:43.492239 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:43.492119    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:43.492239 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:43.492181    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:44.491704 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:44.491669    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:44.492132 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:44.491771    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:45.491581 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:45.491537    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:45.491581 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:45.491570    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:45.492078 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:45.491667    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:45.492078 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:45.491778    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:46.163253 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:46.163210    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:18:46.163451 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:46.163435    2569 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:18:46.174041 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:46.174001    2569 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" podUID="76ab3954-a12f-484a-b32f-3de7517bc70f" containerName="ovnkube-controller" probeResult="failure" output=""
Oct 30 22:18:46.183430 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:46.183398    2569 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75" podUID="76ab3954-a12f-484a-b32f-3de7517bc70f" containerName="ovnkube-controller" probeResult="failure" output=""
Oct 30 22:18:46.492086 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:46.492021    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:46.492453 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:46.492114    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:47.491545 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:47.491512    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:47.491715 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:47.491512    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:47.491715 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:47.491637    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:47.491715 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:47.491698    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:48.494193 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:48.494165    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:48.494570 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:48.494261    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:49.491332 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:49.491296    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:49.491332 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:49.491326    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:49.491541 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:49.491412    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:49.491576 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:49.491536    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:50.491952 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:50.491919    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:50.492343 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:50.492027    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:51.491692 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:51.491665    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:51.491859 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:51.491667    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:51.491859 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:51.491765    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:51.491944 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:51.491864    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:52.091814 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:52.091783    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:52.091814 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:52.091820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:52.092253 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:52.091951    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:52.092253 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:52.092004    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret podName:1a882ca4-1b11-494c-a51e-fb3c93ef5606 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:24.091989986 +0000 UTC m=+66.172576839 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret") pod "global-pull-secret-syncer-bzxzm" (UID: "1a882ca4-1b11-494c-a51e-fb3c93ef5606") : object "kube-system"/"original-pull-secret" not registered
Oct 30 22:18:52.092253 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:52.091992    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:52.092253 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:52.092053    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs podName:7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa nodeName:}" failed. No retries permitted until 2026-10-30 22:19:24.092039096 +0000 UTC m=+66.172625953 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs") pod "network-metrics-daemon-cdmr4" (UID: "7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 22:18:52.192141 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:52.192112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:52.192310 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:52.192256    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 22:18:52.192310 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:52.192271    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 22:18:52.192310 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:52.192280    2569 projected.go:196] Error preparing data for projected volume kube-api-access-xrdzx for pod openshift-network-diagnostics/network-check-target-27nh7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:52.192417 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:52.192329    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx podName:3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:24.192314391 +0000 UTC m=+66.272901264 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrdzx" (UniqueName: "kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx") pod "network-check-target-27nh7" (UID: "3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 22:18:52.491695 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:52.491651    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:52.491883 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:52.491753    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:53.491581 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:53.491540    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:53.491581 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:53.491569    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:53.492060 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:53.491673    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:53.492060 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:53.491787    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:54.491340 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:54.491310    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:54.491546 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:54.491418    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:55.398613 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:55.398582    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-27nh7"]
Oct 30 22:18:55.399052 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:55.398685    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:55.399052 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:55.398776    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:55.402361 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:55.402330    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bzxzm"]
Oct 30 22:18:55.402489 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:55.402398    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cdmr4"]
Oct 30 22:18:55.402489 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:55.402481    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:55.402609 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:55.402587    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:55.403176 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:55.403145    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:55.403480 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:55.403452    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:57.491612 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:57.491571    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:57.492168 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:57.491620    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:57.492168 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:57.491587    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:57.492168 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:57.491715    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:57.492168 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:57.491824    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:57.492168 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:57.491883    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:59.491917 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.491740    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:18:59.492268 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.491802    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:18:59.492268 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:59.492020    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdmr4" podUID="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa"
Oct 30 22:18:59.492268 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.491810    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:18:59.492268 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:59.492072    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bzxzm" podUID="1a882ca4-1b11-494c-a51e-fb3c93ef5606"
Oct 30 22:18:59.492268 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:18:59.492157    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-27nh7" podUID="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04"
Oct 30 22:18:59.706319 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.706293    2569 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeReady"
Oct 30 22:18:59.706490 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.706421    2569 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 22:18:59.773629 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.773594    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wghlv"]
Oct 30 22:18:59.793471 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.793427    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gt7k2"]
Oct 30 22:18:59.793646 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.793582    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wghlv"
Oct 30 22:18:59.801759 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.801733    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 22:18:59.801935 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.801861    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9jqsq\""
Oct 30 22:18:59.802416 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.801940    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 22:18:59.811516 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.811496    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wghlv"]
Oct 30 22:18:59.811606 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.811579    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gt7k2"]
Oct 30 22:18:59.811644 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.811611    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6w8dk"]
Oct 30 22:18:59.811680 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.811584    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:18:59.825623 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.825603    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6w8dk"]
Oct 30 22:18:59.825732 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.825717    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:18:59.900813 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.900780    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 22:18:59.900813 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.900808    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 22:18:59.901073 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.900844    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 22:18:59.901073 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.900881    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 22:18:59.901073 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.900883    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 22:18:59.901073 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.900936    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-srblb\""
Oct 30 22:18:59.901073 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.900940    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 22:18:59.901073 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.900956    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-jfcbv\""
Oct 30 22:18:59.901073 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.900994    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 22:18:59.945373 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945345    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7110fffb-3090-497a-a6d8-0260f3166ed1-metrics-tls\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:18:59.945470 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945392    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb2jv\" (UniqueName: \"kubernetes.io/projected/7110fffb-3090-497a-a6d8-0260f3166ed1-kube-api-access-kb2jv\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:18:59.945470 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945415    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:18:59.945554 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945465    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7110fffb-3090-497a-a6d8-0260f3166ed1-config-volume\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:18:59.945588 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945554    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9eaa777-0ec3-4c87-92a6-05e46f50066f-crio-socket\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:18:59.945649 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945635    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9eaa777-0ec3-4c87-92a6-05e46f50066f-data-volume\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:18:59.945683 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945655    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7110fffb-3090-497a-a6d8-0260f3166ed1-tmp-dir\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:18:59.945683 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eaa777-0ec3-4c87-92a6-05e46f50066f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:18:59.945752 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945694    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60e9435e-1582-493b-a73d-226b32c86abc-cert\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:18:59.945752 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945737    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m8hc\" (UniqueName: \"kubernetes.io/projected/60e9435e-1582-493b-a73d-226b32c86abc-kube-api-access-4m8hc\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:18:59.945823 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:18:59.945758    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nspxc\" (UniqueName: \"kubernetes.io/projected/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-api-access-nspxc\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.046366 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046297    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eaa777-0ec3-4c87-92a6-05e46f50066f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.046366 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60e9435e-1582-493b-a73d-226b32c86abc-cert\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:19:00.046366 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046365    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m8hc\" (UniqueName: \"kubernetes.io/projected/60e9435e-1582-493b-a73d-226b32c86abc-kube-api-access-4m8hc\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:19:00.046590 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046388    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nspxc\" (UniqueName: \"kubernetes.io/projected/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-api-access-nspxc\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.046590 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7110fffb-3090-497a-a6d8-0260f3166ed1-metrics-tls\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:00.046590 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046468    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb2jv\" (UniqueName: \"kubernetes.io/projected/7110fffb-3090-497a-a6d8-0260f3166ed1-kube-api-access-kb2jv\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:00.046590 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046501    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.046590 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046523    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7110fffb-3090-497a-a6d8-0260f3166ed1-config-volume\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:00.046590 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046548    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9eaa777-0ec3-4c87-92a6-05e46f50066f-crio-socket\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.046878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046602    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9eaa777-0ec3-4c87-92a6-05e46f50066f-data-volume\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.046878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046626    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7110fffb-3090-497a-a6d8-0260f3166ed1-tmp-dir\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:00.046878 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.046872    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9eaa777-0ec3-4c87-92a6-05e46f50066f-crio-socket\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.047049 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.047027    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9eaa777-0ec3-4c87-92a6-05e46f50066f-data-volume\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.047125 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.047103    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7110fffb-3090-497a-a6d8-0260f3166ed1-tmp-dir\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:00.047227 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.047205    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.047275 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.047229    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7110fffb-3090-497a-a6d8-0260f3166ed1-config-volume\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:00.050470 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.050440    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eaa777-0ec3-4c87-92a6-05e46f50066f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.050470 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.050459    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7110fffb-3090-497a-a6d8-0260f3166ed1-metrics-tls\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:00.050622 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.050485    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60e9435e-1582-493b-a73d-226b32c86abc-cert\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:19:00.059616 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.059590    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nspxc\" (UniqueName: \"kubernetes.io/projected/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-api-access-nspxc\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.059819 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.059802    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m8hc\" (UniqueName: \"kubernetes.io/projected/60e9435e-1582-493b-a73d-226b32c86abc-kube-api-access-4m8hc\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:19:00.059891 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.059824    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb2jv\" (UniqueName: \"kubernetes.io/projected/7110fffb-3090-497a-a6d8-0260f3166ed1-kube-api-access-kb2jv\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:00.132659 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.132620    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:00.147815 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.147790    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:19:00.188087 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.188066    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:00.497578 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.497559    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6w8dk"]
Oct 30 22:19:00.498109 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.497627    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gt7k2"]
Oct 30 22:19:00.498109 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.497652    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wghlv"]
Oct 30 22:19:00.499353 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:00.499330    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9eaa777_0ec3_4c87_92a6_05e46f50066f.slice/crio-fba94a278e13d0b321e64c666787eb0e317aed77974fbf8ff88d74b7dda6ba24 WatchSource:0}: Error finding container fba94a278e13d0b321e64c666787eb0e317aed77974fbf8ff88d74b7dda6ba24: Status 404 returned error can't find the container with id fba94a278e13d0b321e64c666787eb0e317aed77974fbf8ff88d74b7dda6ba24
Oct 30 22:19:00.500242 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:00.500204    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60e9435e_1582_493b_a73d_226b32c86abc.slice/crio-e23ef180860396c0b4899f2c6a7b424655b12cf8f073880fd8d2e5083cf8a563 WatchSource:0}: Error finding container e23ef180860396c0b4899f2c6a7b424655b12cf8f073880fd8d2e5083cf8a563: Status 404 returned error can't find the container with id e23ef180860396c0b4899f2c6a7b424655b12cf8f073880fd8d2e5083cf8a563
Oct 30 22:19:00.500874 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:00.500850    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7110fffb_3090_497a_a6d8_0260f3166ed1.slice/crio-2198be6303a3f26b23867e49661a3fb6af743c5d8799fed8d13a7bad12d4a3cb WatchSource:0}: Error finding container 2198be6303a3f26b23867e49661a3fb6af743c5d8799fed8d13a7bad12d4a3cb: Status 404 returned error can't find the container with id 2198be6303a3f26b23867e49661a3fb6af743c5d8799fed8d13a7bad12d4a3cb
Oct 30 22:19:00.791033 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.790771    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6w8dk" event={"ID":"f9eaa777-0ec3-4c87-92a6-05e46f50066f","Type":"ContainerStarted","Data":"c9de258d6ba11825a4b099499ea04032b1dd1ed1c2d1f1bde437b1af09f49dd9"}
Oct 30 22:19:00.791033 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.790997    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6w8dk" event={"ID":"f9eaa777-0ec3-4c87-92a6-05e46f50066f","Type":"ContainerStarted","Data":"fba94a278e13d0b321e64c666787eb0e317aed77974fbf8ff88d74b7dda6ba24"}
Oct 30 22:19:00.791884 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.791861    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gt7k2" event={"ID":"60e9435e-1582-493b-a73d-226b32c86abc","Type":"ContainerStarted","Data":"e23ef180860396c0b4899f2c6a7b424655b12cf8f073880fd8d2e5083cf8a563"}
Oct 30 22:19:00.792956 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:00.792937    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wghlv" event={"ID":"7110fffb-3090-497a-a6d8-0260f3166ed1","Type":"ContainerStarted","Data":"2198be6303a3f26b23867e49661a3fb6af743c5d8799fed8d13a7bad12d4a3cb"}
Oct 30 22:19:01.491209 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.491174    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:19:01.491625 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.491607    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:19:01.492030 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.492013    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:01.496404 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.496321    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 22:19:01.496662 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.496632    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 22:19:01.497913 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.497870    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-sdfwl\""
Oct 30 22:19:01.498225 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.497948    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms5nl\""
Oct 30 22:19:01.498824 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.498807    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 22:19:01.498892 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.498869    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 22:19:01.498964 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:01.498956    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-r4dw4\""
Oct 30 22:19:02.007544 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.007519    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"]
Oct 30 22:19:02.024657 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.024632    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.027201 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.027173    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"]
Oct 30 22:19:02.036236 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.036195    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 22:19:02.036363 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.036343    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 22:19:02.036504 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.036490    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 22:19:02.036988 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.036966    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 22:19:02.037283 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.037256    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 22:19:02.037439 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.037423    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-cg9xt\""
Oct 30 22:19:02.047223 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.047201    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2j5mw"]
Oct 30 22:19:02.061751 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.061716    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"]
Oct 30 22:19:02.062726 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.062701    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.075392 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.075367    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"]
Oct 30 22:19:02.075503 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.075488    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.104990 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.104833    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 22:19:02.104990 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.104866    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 22:19:02.104990 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.104919    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bh4rp\""
Oct 30 22:19:02.105300 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.105281    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 22:19:02.105403 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.105311    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 22:19:02.105403 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.105344    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-m6d59\""
Oct 30 22:19:02.129857 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.129787    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 22:19:02.130176 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.130155    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 22:19:02.162352 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162319    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.162495 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162366    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.162495 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162394    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-wtmp\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.162495 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162420    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-metrics-client-ca\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.162495 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162445    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvw67\" (UniqueName: \"kubernetes.io/projected/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-kube-api-access-mvw67\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.162495 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-root\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.162684 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162519    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.162684 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162545    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.162684 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162572    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.162684 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162598    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw7st\" (UniqueName: \"kubernetes.io/projected/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-kube-api-access-xw7st\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.162684 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162656    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.162858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162683    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.162858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.162858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162738    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.162858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162763    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gng5d\" (UniqueName: \"kubernetes.io/projected/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-api-access-gng5d\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.162858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162789    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-sys\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.162858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-tls\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.162858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162835    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.162858 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.162857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-textfile\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.263952 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.263889    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-textfile\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.264113 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264000    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.264113 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264032    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.264113 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264060    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-wtmp\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.264113 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264084    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-metrics-client-ca\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.264113 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvw67\" (UniqueName: \"kubernetes.io/projected/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-kube-api-access-mvw67\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.264338 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264154    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-root\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.264338 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264184    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.264338 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264207    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.264338 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264234    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.264338 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264256    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-textfile\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.264338 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264260    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw7st\" (UniqueName: \"kubernetes.io/projected/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-kube-api-access-xw7st\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.264560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264359    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.264560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.264560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.264560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264432    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.264560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264458    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gng5d\" (UniqueName: \"kubernetes.io/projected/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-api-access-gng5d\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.264560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264474    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-sys\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.264560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264496    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-tls\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.264560 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.264515    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.265343 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.265322    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.265957 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.265882    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.265957 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.265926    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.266685 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.266663    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.266794 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.266776    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-wtmp\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.266997 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.266974    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-sys\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.267293 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.267268    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-metrics-client-ca\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.267385 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.267364    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-root\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.267574 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.267547    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.269197 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.269025    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.269197 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.269123    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.270017 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.269992    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-tls\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.271381 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.271353    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.271501 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.271477    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.271691 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.271669    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.279069 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.279047    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gng5d\" (UniqueName: \"kubernetes.io/projected/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-api-access-gng5d\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.279161 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.279096    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw7st\" (UniqueName: \"kubernetes.io/projected/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-kube-api-access-xw7st\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.280670 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.280645    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvw67\" (UniqueName: \"kubernetes.io/projected/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-kube-api-access-mvw67\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.376301 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.376253    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:02.391494 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.391424    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:02.433406 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:02.433370    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77422c7f_1d2c_4ca8_968c_3e59beffbb1e.slice/crio-57e264ca0570959ff62694180f54ea975cfef446444aab0a3e139e9a48e7faec WatchSource:0}: Error finding container 57e264ca0570959ff62694180f54ea975cfef446444aab0a3e139e9a48e7faec: Status 404 returned error can't find the container with id 57e264ca0570959ff62694180f54ea975cfef446444aab0a3e139e9a48e7faec
Oct 30 22:19:02.435317 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.435293    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:02.687432 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.687401    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"]
Oct 30 22:19:02.790345 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.790305    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"]
Oct 30 22:19:02.795163 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:02.795133    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8cf69d_9ab1_4bfa_b927_509c0e847d1a.slice/crio-99392956d77083b97274fcae86e3dba3c838d69117f3d473c2c3d6f12573bf49 WatchSource:0}: Error finding container 99392956d77083b97274fcae86e3dba3c838d69117f3d473c2c3d6f12573bf49: Status 404 returned error can't find the container with id 99392956d77083b97274fcae86e3dba3c838d69117f3d473c2c3d6f12573bf49
Oct 30 22:19:02.804257 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.804225    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2j5mw" event={"ID":"77422c7f-1d2c-4ca8-968c-3e59beffbb1e","Type":"ContainerStarted","Data":"57e264ca0570959ff62694180f54ea975cfef446444aab0a3e139e9a48e7faec"}
Oct 30 22:19:02.806374 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.806342    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6w8dk" event={"ID":"f9eaa777-0ec3-4c87-92a6-05e46f50066f","Type":"ContainerStarted","Data":"8f9d918a74db8c8425d42d2331c5bfefd7467608737052a1f69bd3480b1b6a3f"}
Oct 30 22:19:02.809274 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.809230    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz" event={"ID":"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a","Type":"ContainerStarted","Data":"99392956d77083b97274fcae86e3dba3c838d69117f3d473c2c3d6f12573bf49"}
Oct 30 22:19:02.811272 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.811249    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8" event={"ID":"2fb7c085-6b91-409e-b15b-6d5b94f1c94c","Type":"ContainerStarted","Data":"8d23da89396e0eb3763a357b85fe9e4bb4cdae904ab8265915f2ecb014d79cd0"}
Oct 30 22:19:02.863327 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.863300    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-688b666f4d-5pmrq"]
Oct 30 22:19:02.889592 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.889481    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688b666f4d-5pmrq"]
Oct 30 22:19:02.889663 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.889616    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:02.895563 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.895537    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 22:19:02.895667 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.895602    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 22:19:02.895724 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.895675    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-gb6fw\""
Oct 30 22:19:02.896209 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.896011    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 22:19:02.896209 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.896090    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 22:19:02.896209 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.896103    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 22:19:02.898587 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.898567    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 22:19:02.967159 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.966936    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 22:19:02.970134 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.970103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:02.970234 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.970152    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:02.970234 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.970181    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:02.970234 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.970213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:02.970393 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.970282    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:02.970393 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.970317    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:02.970393 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:02.970363    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.070781    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.070850    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.070874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.070918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.070954    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.070993    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.071068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.071539    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.071762    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072108 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.071780    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.072654 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.072388    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.077704 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.077676    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.078664 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.078628    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.085340 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.085316    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.152008 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.151980    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:19:03.172838 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.172808    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.182972 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.182947    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:19:03.183819 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.183759    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 22:19:03.184106 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.184066    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 22:19:03.184800 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.184408    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 22:19:03.185044 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.185026    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 22:19:03.186984 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.185073    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 22:19:03.186984 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.185133    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 22:19:03.186984 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.185187    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 22:19:03.186984 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.185233    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x6d62\""
Oct 30 22:19:03.186984 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.186332    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 22:19:03.186984 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.186378    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 22:19:03.228890 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.228780    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272512    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272635    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272664    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272698    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272723    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272747    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2vgr\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272786    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272837    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272938    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.273258 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.272973    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374255 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374221    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374424 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374266    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374424 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374298    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374424 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374328    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374424 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374352    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374424 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374379    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374424 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374414    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374718 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374718 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374470    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vgr\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374718 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374506    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374718 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374597    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374718 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374668    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.374718 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.374700    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.376655 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.376521    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.378346 ip-10-0-132-192 kubenswrapper[2569]: E1030 22:19:03.377733    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle podName:08103390-6605-487e-b4bb-9e367e7dda1f nodeName:}" failed. No retries permitted until 2026-10-30 22:19:03.877708467 +0000 UTC m=+45.958295355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f") : configmap references non-existent config key: ca-bundle.crt
Oct 30 22:19:03.378581 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.378563    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.378730 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.378706    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.378805 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.378752    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.380496 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.380456    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.381775 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.381755    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.383539 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.383514    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.383635 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.383582    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.384314 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.384024    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.385726 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.385593    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.389366 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.389343    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vgr\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.393545 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.393492    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.553288 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.553161    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688b666f4d-5pmrq"]
Oct 30 22:19:03.557230 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:03.557196    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca5a6fe7_f3f9_46eb_a4f5_f113c89e0c28.slice/crio-fcdb921f5a039678d98a0e65e7a92593630132e11262305e0b58767321407e56 WatchSource:0}: Error finding container fcdb921f5a039678d98a0e65e7a92593630132e11262305e0b58767321407e56: Status 404 returned error can't find the container with id fcdb921f5a039678d98a0e65e7a92593630132e11262305e0b58767321407e56
Oct 30 22:19:03.817528 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.817443    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8" event={"ID":"2fb7c085-6b91-409e-b15b-6d5b94f1c94c","Type":"ContainerStarted","Data":"f157558bd531d4d6d4cce4be56624d5c4691d64500e7f866bf293aead811a95d"}
Oct 30 22:19:03.817528 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.817518    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8" event={"ID":"2fb7c085-6b91-409e-b15b-6d5b94f1c94c","Type":"ContainerStarted","Data":"fd3c8010b8fcbb3b6a5ac9e68db50d11bdcdf9314b2840ed1fea5ff21d68135f"}
Oct 30 22:19:03.819923 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.819875    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688b666f4d-5pmrq" event={"ID":"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28","Type":"ContainerStarted","Data":"fcdb921f5a039678d98a0e65e7a92593630132e11262305e0b58767321407e56"}
Oct 30 22:19:03.823579 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.823553    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wghlv" event={"ID":"7110fffb-3090-497a-a6d8-0260f3166ed1","Type":"ContainerStarted","Data":"1620fd02c6b4b3c9361993afd6dc50521ca941951a34e892f31c54974d918abc"}
Oct 30 22:19:03.823679 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.823604    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wghlv" event={"ID":"7110fffb-3090-497a-a6d8-0260f3166ed1","Type":"ContainerStarted","Data":"c8389cd87fd080fc98cb5b37884a1bdb1f5da7a89c973a743de43faf7d8ea3a6"}
Oct 30 22:19:03.824014 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.823984    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:03.864632 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.864038    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-wghlv" podStartSLOduration=2.549269816 podStartE2EDuration="4.864019016s" totalImagesPullingTime="2.3147492s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:03.863010086 +0000 UTC m=+45.943596967" watchObservedRunningTime="2026-10-30 22:19:03.864019016 +0000 UTC m=+45.944605892"
Oct 30 22:19:03.879387 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.879351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.880325 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.880297    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:03.882589 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:03.882568    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:04.264168 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:04.264133    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:19:04.829700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:04.829459    2569 generic.go:356] "Generic (PLEG): container finished" podID="77422c7f-1d2c-4ca8-968c-3e59beffbb1e" containerID="cede5a467a81b66a674000268cdbb4fccae9ecf5b8918f3cbf077f7805524d39" exitCode=0
Oct 30 22:19:04.829700 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:04.829538    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2j5mw" event={"ID":"77422c7f-1d2c-4ca8-968c-3e59beffbb1e","Type":"ContainerDied","Data":"cede5a467a81b66a674000268cdbb4fccae9ecf5b8918f3cbf077f7805524d39"}
Oct 30 22:19:04.831073 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:04.831050    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerStarted","Data":"3a3dec3c50c3664399b01d817628bf0151f02624a2be2a70e3905315c1338337"}
Oct 30 22:19:04.832740 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:04.832693    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gt7k2" event={"ID":"60e9435e-1582-493b-a73d-226b32c86abc","Type":"ContainerStarted","Data":"efd8e0464f9d74e3a821e5bfcb89913a9961bba9c4016dce5288388dda01901e"}
Oct 30 22:19:04.835244 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:04.835183    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6w8dk" event={"ID":"f9eaa777-0ec3-4c87-92a6-05e46f50066f","Type":"ContainerStarted","Data":"362f7a2403547a6c847ae0c9bb381c1a21412c153a0a58a36767588b53be48bd"}
Oct 30 22:19:04.877757 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:04.877678    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gt7k2" podStartSLOduration=2.407093097 podStartE2EDuration="5.877662311s" totalImagesPullingTime="3.470569214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:04.877293625 +0000 UTC m=+46.957880501" watchObservedRunningTime="2026-10-30 22:19:04.877662311 +0000 UTC m=+46.958249186"
Oct 30 22:19:04.899563 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:04.899516    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6w8dk" podStartSLOduration=2.312019262 podStartE2EDuration="5.899503918s" totalImagesPullingTime="3.587484656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 22:18:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:04.899209538 +0000 UTC m=+46.979796416" watchObservedRunningTime="2026-10-30 22:19:04.899503918 +0000 UTC m=+46.980090793"
Oct 30 22:19:05.842623 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:05.842442    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2j5mw" event={"ID":"77422c7f-1d2c-4ca8-968c-3e59beffbb1e","Type":"ContainerStarted","Data":"69f8da2bda1c40d232cfd77a8e1afece7f49af24fa8d913aecf490912500088d"}
Oct 30 22:19:05.842623 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:05.842513    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2j5mw" event={"ID":"77422c7f-1d2c-4ca8-968c-3e59beffbb1e","Type":"ContainerStarted","Data":"f1d76e9b4a876f3802e717fdcb102f8fddf315c95da0a959b1bd1dac7765a67b"}
Oct 30 22:19:05.845133 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:05.845104    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz" event={"ID":"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a","Type":"ContainerStarted","Data":"dcbd57ae14fd1a320480b854dc1e79690c6f8f9434ada7afed77081ff8d6ff81"}
Oct 30 22:19:05.845257 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:05.845164    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz" event={"ID":"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a","Type":"ContainerStarted","Data":"44259f004d2f11993d978a693bf06301d72d94358ce292648362ec88e25d7104"}
Oct 30 22:19:05.845257 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:05.845193    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz" event={"ID":"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a","Type":"ContainerStarted","Data":"3d709ce7b20738420f1fee5d5a71e9edf25927e0b8e53e0efd74c1fc47669c71"}
Oct 30 22:19:05.849691 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:05.849666    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8" event={"ID":"2fb7c085-6b91-409e-b15b-6d5b94f1c94c","Type":"ContainerStarted","Data":"b74f4aa6879aa8e76e4be8b15103252c85f2980209af92833610a8df49b9beaa"}
Oct 30 22:19:05.867358 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:05.867299    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2j5mw" podStartSLOduration=1.868683959 podStartE2EDuration="3.867281669s" totalImagesPullingTime="1.891243457s" totalInitContainerRuntime="107.354253ms" isStatefulPod=true podCreationTimestamp="2026-10-30 22:19:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:05.866460744 +0000 UTC m=+47.947047613" watchObservedRunningTime="2026-10-30 22:19:05.867281669 +0000 UTC m=+47.947868545"
Oct 30 22:19:05.889606 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:05.889542    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz" podStartSLOduration=1.5525583950000001 podStartE2EDuration="3.889523281s" totalImagesPullingTime="2.336964886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:05.887760638 +0000 UTC m=+47.968347514" watchObservedRunningTime="2026-10-30 22:19:05.889523281 +0000 UTC m=+47.970110156"
Oct 30 22:19:05.915489 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:05.915430    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8" podStartSLOduration=2.7966659480000002 podStartE2EDuration="4.915409782s" totalImagesPullingTime="2.118743834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:05.914073553 +0000 UTC m=+47.994660429" watchObservedRunningTime="2026-10-30 22:19:05.915409782 +0000 UTC m=+47.995996653"
Oct 30 22:19:06.854842 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:06.854805    2569 generic.go:356] "Generic (PLEG): container finished" podID="08103390-6605-487e-b4bb-9e367e7dda1f" containerID="8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e" exitCode=0
Oct 30 22:19:06.855296 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:06.854935    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerDied","Data":"8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e"}
Oct 30 22:19:07.861043 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:07.861008    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688b666f4d-5pmrq" event={"ID":"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28","Type":"ContainerStarted","Data":"7e988ab5297b90484d1859c2681f17a7f44415615e746a0b8e44435594935283"}
Oct 30 22:19:07.885121 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:07.885076    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-688b666f4d-5pmrq" podStartSLOduration=2.519779883 podStartE2EDuration="5.885058044s" totalImagesPullingTime="3.365278161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:07.883059217 +0000 UTC m=+49.963646093" watchObservedRunningTime="2026-10-30 22:19:07.885058044 +0000 UTC m=+49.965644984"
Oct 30 22:19:08.470770 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.470741    2569 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:19:08.478934 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.478252    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.486977 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.486955    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 22:19:08.487076 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.486994    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 22:19:08.487130 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.487118    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 22:19:08.487291 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.487276    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 22:19:08.487458 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.487440    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 22:19:08.487513 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.487458    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 22:19:08.487800 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.487788    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 22:19:08.487982 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.487971    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4f76b\""
Oct 30 22:19:08.488723 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.488709    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 22:19:08.488785 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.488737    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 22:19:08.490127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.490103    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 22:19:08.490243 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.490225    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 22:19:08.490243 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.490228    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 22:19:08.491017 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.490963    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-caus6766sf7av\""
Oct 30 22:19:08.491090 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.491040    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 22:19:08.494562 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.494541    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 22:19:08.496024 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.495984    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 22:19:08.499335 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.499310    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:19:08.625831 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.625796    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.625977 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.625837    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnlcg\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.625977 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.625945    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626063 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.625987    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626063 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626045    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626139 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626139 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626139 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626133    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626276 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626153    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626276 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626252    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626380 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626380 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626467 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626467 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626450    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626590 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626475    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626590 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626590 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626509    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626677 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626586    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626677 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626661    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.626744 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.626685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.727789 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.727755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.727789 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.727794    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728064 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.727957    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728064 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728064 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728031    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728064 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728059    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728248 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728300 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728251    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728384 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728364    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728444 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728401    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnlcg\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728497 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728442    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728497 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728478    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728607 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728503    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728607 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728531    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728607 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728556    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728607 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728585    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728779 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728611    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728779 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728645    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728779 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728678    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.728779 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728711    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.729160 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728960    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.729425 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.729399    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.729509 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.729485    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.731610 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.731512    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.731610 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.728501    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.732001 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.731885    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.732104 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.732012    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.732328 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.732238    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.732328 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.732268    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.732610 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.732584    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.732688 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.732659    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.732848 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.732804    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.733055 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.733030    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.733127 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.733037    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.733172 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.733120    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.733701 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.733682    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.734196 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.734176    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.734293 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.734277    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.735298 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.735282    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.746264 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.746214    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnlcg\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.817023 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.816998    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:08.872843 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.872814    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerStarted","Data":"591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11"}
Oct 30 22:19:08.873242 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.872875    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerStarted","Data":"74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351"}
Oct 30 22:19:08.873242 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.872930    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerStarted","Data":"cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476"}
Oct 30 22:19:08.873242 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.872955    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerStarted","Data":"8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef"}
Oct 30 22:19:08.873242 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:08.872976    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerStarted","Data":"fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012"}
Oct 30 22:19:09.108424 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:09.108346    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:19:09.112884 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:09.112858    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode92a79fb_ac06_45b2_b606_5fb4eabf94d7.slice/crio-7539f30ff407b8642f151d28b324a4fb27c751a44529fb60cc780195b0e4d274 WatchSource:0}: Error finding container 7539f30ff407b8642f151d28b324a4fb27c751a44529fb60cc780195b0e4d274: Status 404 returned error can't find the container with id 7539f30ff407b8642f151d28b324a4fb27c751a44529fb60cc780195b0e4d274
Oct 30 22:19:09.877341 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:09.877251    2569 generic.go:356] "Generic (PLEG): container finished" podID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc" exitCode=0
Oct 30 22:19:09.877341 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:09.877292    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerDied","Data":"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"}
Oct 30 22:19:09.877813 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:09.877423    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerStarted","Data":"7539f30ff407b8642f151d28b324a4fb27c751a44529fb60cc780195b0e4d274"}
Oct 30 22:19:09.881067 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:09.881045    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerStarted","Data":"af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e"}
Oct 30 22:19:09.947532 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:09.947482    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.040387921 podStartE2EDuration="6.94746857s" totalImagesPullingTime="3.892545414s" totalInitContainerRuntime="14.535235ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:09.946157203 +0000 UTC m=+52.026744079" watchObservedRunningTime="2026-10-30 22:19:09.94746857 +0000 UTC m=+52.028055442"
Oct 30 22:19:12.897327 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:12.897297    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerStarted","Data":"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"}
Oct 30 22:19:12.897625 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:12.897381    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerStarted","Data":"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"}
Oct 30 22:19:13.229557 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:13.229518    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:13.229919 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:13.229603    2569 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:13.235649 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:13.235627    2569 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:13.837918 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:13.837868    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:13.906699 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:13.906668    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:14.912324 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:14.912296    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerStarted","Data":"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"}
Oct 30 22:19:15.918827 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:15.918789    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerStarted","Data":"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"}
Oct 30 22:19:15.919202 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:15.918859    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerStarted","Data":"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"}
Oct 30 22:19:15.919202 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:15.918888    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerStarted","Data":"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"}
Oct 30 22:19:15.960488 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:15.960426    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.119031908 podStartE2EDuration="7.960407885s" totalImagesPullingTime="4.820574795s" totalInitContainerRuntime="20.801182ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:15.959042518 +0000 UTC m=+58.039629408" watchObservedRunningTime="2026-10-30 22:19:15.960407885 +0000 UTC m=+58.040994763"
Oct 30 22:19:16.184681 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:16.184608    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:18.817981 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:18.817946    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:19.037793 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:19.037753    2569 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688b666f4d-5pmrq"]
Oct 30 22:19:24.162148 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.162109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:19:24.162148 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.162148    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:24.165025 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.165009    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 22:19:24.165874 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.165858    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 22:19:24.175212 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.175187    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:24.175319 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.175302    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:19:24.262512 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.262482    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:19:24.265298 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.265277    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 22:19:24.275264 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.275246    2569 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 22:19:24.285341 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.285320    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:19:24.340701 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.340680    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms5nl\""
Oct 30 22:19:24.348753 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.348737    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:19:24.361649 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.361629    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-r4dw4\""
Oct 30 22:19:24.368945 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.368927    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:24.382766 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.382743    2569 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-sdfwl\""
Oct 30 22:19:24.390076 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.389987    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:19:24.632667 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.632638    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cdmr4"]
Oct 30 22:19:24.635115 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:24.635089    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e6b9a4a_5287_4baf_a5e2_1de5f7b9befa.slice/crio-2279054b88b2fe10242e46ba096b251fcb0dffc9fc308fae535ccfac8053dd17 WatchSource:0}: Error finding container 2279054b88b2fe10242e46ba096b251fcb0dffc9fc308fae535ccfac8053dd17: Status 404 returned error can't find the container with id 2279054b88b2fe10242e46ba096b251fcb0dffc9fc308fae535ccfac8053dd17
Oct 30 22:19:24.656084 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.656062    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bzxzm"]
Oct 30 22:19:24.658312 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:24.658279    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a882ca4_1b11_494c_a51e_fb3c93ef5606.slice/crio-4f7a9b4759fb18f1b558d4f329f9ee200a2d4ce5526c9f4d96055dc85a45ff3d WatchSource:0}: Error finding container 4f7a9b4759fb18f1b558d4f329f9ee200a2d4ce5526c9f4d96055dc85a45ff3d: Status 404 returned error can't find the container with id 4f7a9b4759fb18f1b558d4f329f9ee200a2d4ce5526c9f4d96055dc85a45ff3d
Oct 30 22:19:24.684729 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.684693    2569 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-27nh7"]
Oct 30 22:19:24.687227 ip-10-0-132-192 kubenswrapper[2569]: W1030 22:19:24.687202    2569 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea2d4d8_a3e0_4865_ae75_9db94e8c2d04.slice/crio-7821eabf282976a84916b1655ea63d0434909eecbf1475fcee413a7057f48df3 WatchSource:0}: Error finding container 7821eabf282976a84916b1655ea63d0434909eecbf1475fcee413a7057f48df3: Status 404 returned error can't find the container with id 7821eabf282976a84916b1655ea63d0434909eecbf1475fcee413a7057f48df3
Oct 30 22:19:24.951003 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.950916    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bzxzm" event={"ID":"1a882ca4-1b11-494c-a51e-fb3c93ef5606","Type":"ContainerStarted","Data":"4f7a9b4759fb18f1b558d4f329f9ee200a2d4ce5526c9f4d96055dc85a45ff3d"}
Oct 30 22:19:24.952047 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.952019    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cdmr4" event={"ID":"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa","Type":"ContainerStarted","Data":"2279054b88b2fe10242e46ba096b251fcb0dffc9fc308fae535ccfac8053dd17"}
Oct 30 22:19:24.953028 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:24.953010    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-27nh7" event={"ID":"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04","Type":"ContainerStarted","Data":"7821eabf282976a84916b1655ea63d0434909eecbf1475fcee413a7057f48df3"}
Oct 30 22:19:26.965766 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:26.964948    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cdmr4" event={"ID":"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa","Type":"ContainerStarted","Data":"00956a148bd7dc955201b1ddec5f74c203b47365ce1e462af1e0125b700ce9b6"}
Oct 30 22:19:26.965766 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:26.965012    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cdmr4" event={"ID":"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa","Type":"ContainerStarted","Data":"01a7e3e4b7bdd6894d503a45a6f74a40350084eafa93aebb0b5650e49526e40a"}
Oct 30 22:19:26.986973 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:26.986918    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cdmr4" podStartSLOduration=66.768241703 podStartE2EDuration="1m7.986877897s" totalImagesPullingTime="1.218636194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:26.985434512 +0000 UTC m=+69.066021388" watchObservedRunningTime="2026-10-30 22:19:26.986877897 +0000 UTC m=+69.067464773"
Oct 30 22:19:28.317875 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:28.317783    2569 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:28.340050 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:28.340024    2569 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:28.835488 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:28.835458    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:28.977083 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:28.977040    2569 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-27nh7" event={"ID":"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04","Type":"ContainerStarted","Data":"4cde5bcbb074650df3d3006e92a8fb9e3a7dba80cb92682eb254ccf5cabbdb5c"}
Oct 30 22:19:28.977455 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:28.977432    2569 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:19:28.996718 ip-10-0-132-192 kubenswrapper[2569]: I1030 22:19:28.996656    2569 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-27nh7" podStartSLOduration=66.642632011 podStartE2EDuration="1m9.996640916s" totalImagesPullingTime="3.354008905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:28.995844095 +0000 UTC m=+71.076430971" watchObservedRunningTime="2026-10-30 22:19:28.996640916 +0000 UTC m=+71.077227790"
Oct 30 22:19:29.682573 ip-10-0-132-192 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 22:19:29.701289 ip-10-0-132-192 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 22:19:29.701457 ip-10-0-132-192 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 22:19:29.701627 ip-10-0-132-192 systemd[1]: kubelet.service: Consumed 4.369s CPU time.
Oct 30 22:19:29.715411 ip-10-0-132-192 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 22:19:29.779056 ip-10-0-132-192 kubenswrapper[5664]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:19:29.779056 ip-10-0-132-192 kubenswrapper[5664]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 22:19:29.779056 ip-10-0-132-192 kubenswrapper[5664]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:19:29.779056 ip-10-0-132-192 kubenswrapper[5664]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 22:19:29.781164 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781077    5664 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:29.781164 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781164    5664 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:29.781164 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781168    5664 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781172    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781175    5664 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781178    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781180    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781183    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781186    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781188    5664 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781190    5664 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781193    5664 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781196    5664 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781208    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781211    5664 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781214    5664 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781216    5664 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781219    5664 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781221    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781224    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781226    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:29.781373 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781229    5664 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781231    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781235    5664 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781237    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781241    5664 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781243    5664 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781247    5664 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781249    5664 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781252    5664 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781254    5664 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781257    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781260    5664 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781262    5664 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781265    5664 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781267    5664 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781270    5664 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781273    5664 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781276    5664 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781278    5664 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:29.781860 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781281    5664 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781284    5664 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781286    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781289    5664 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781292    5664 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781294    5664 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781297    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781300    5664 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781302    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781305    5664 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781308    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781310    5664 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781313    5664 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781315    5664 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781318    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781320    5664 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781323    5664 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781326    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781329    5664 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781332    5664 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:29.782378 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781334    5664 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781336    5664 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781339    5664 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781341    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781344    5664 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781346    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781350    5664 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781352    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781355    5664 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781358    5664 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781360    5664 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781363    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781365    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781368    5664 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781370    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781373    5664 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781375    5664 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781378    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781380    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781384    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:29.782888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781387    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781390    5664 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781392    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781401    5664 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781406    5664 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781409    5664 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781412    5664 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781414    5664 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781417    5664 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781419    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781422    5664 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781424    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781427    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781430    5664 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781433    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781435    5664 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781438    5664 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781442    5664 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781446    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:29.783454 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781449    5664 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781451    5664 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781454    5664 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781456    5664 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781459    5664 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781461    5664 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.781465    5664 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782252    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782259    5664 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782263    5664 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782266    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782269    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782272    5664 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782275    5664 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782278    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782281    5664 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782284    5664 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782286    5664 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782289    5664 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782292    5664 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:29.783963 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782294    5664 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782297    5664 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782299    5664 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782302    5664 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782305    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782307    5664 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782310    5664 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782312    5664 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782315    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782318    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782320    5664 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782323    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782325    5664 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782328    5664 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782330    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782333    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782335    5664 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782337    5664 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782340    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:29.784479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782342    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782345    5664 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782347    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782350    5664 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782353    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782355    5664 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782358    5664 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782361    5664 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782364    5664 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782367    5664 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782372    5664 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782376    5664 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782380    5664 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782383    5664 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782386    5664 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782388    5664 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782391    5664 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782394    5664 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782396    5664 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:29.784995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782399    5664 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782402    5664 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782404    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782407    5664 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782409    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782412    5664 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782415    5664 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782417    5664 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782420    5664 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782422    5664 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782425    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782427    5664 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782430    5664 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782433    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782435    5664 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782438    5664 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782441    5664 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782452    5664 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782455    5664 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782458    5664 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:29.785474 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782460    5664 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782464    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782467    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782469    5664 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782475    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782477    5664 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782479    5664 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782482    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782484    5664 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782487    5664 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782489    5664 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782492    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782494    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782497    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782499    5664 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782502    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782504    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782507    5664 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782509    5664 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:29.785995 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782512    5664 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782516    5664 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782519    5664 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782522    5664 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782525    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782528    5664 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782530    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782533    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782535    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782538    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782541    5664 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782544    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782546    5664 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782549    5664 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782551    5664 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.782560    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783205    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783215    5664 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783219    5664 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:29.786514 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783222    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783225    5664 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783228    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783231    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783234    5664 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783236    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783239    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783241    5664 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783244    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783246    5664 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783249    5664 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783252    5664 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783255    5664 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783257    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783260    5664 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783262    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783265    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783267    5664 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783270    5664 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:29.787023 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783272    5664 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783275    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783278    5664 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783280    5664 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783283    5664 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783285    5664 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783288    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783290    5664 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783293    5664 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783296    5664 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783299    5664 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783303    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783306    5664 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783308    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783312    5664 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783315    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783318    5664 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783321    5664 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783325    5664 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:29.787525 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783328    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783332    5664 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783335    5664 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783338    5664 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783340    5664 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783343    5664 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783346    5664 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783348    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783351    5664 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783354    5664 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783356    5664 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783359    5664 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783361    5664 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783364    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783367    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783370    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783372    5664 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783375    5664 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783377    5664 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:29.788060 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783379    5664 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783382    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783384    5664 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783387    5664 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783390    5664 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783394    5664 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783396    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783399    5664 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783401    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783403    5664 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783406    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783408    5664 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783411    5664 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783414    5664 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783416    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783419    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783421    5664 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783423    5664 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783426    5664 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783428    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:29.788532 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783430    5664 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783437    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783439    5664 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783442    5664 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783444    5664 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783447    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783449    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783452    5664 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783454    5664 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783457    5664 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783459    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783462    5664 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783465    5664 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783467    5664 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783470    5664 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783472    5664 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783475    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783478    5664 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783481    5664 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783484    5664 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:29.789039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783486    5664 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783489    5664 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783491    5664 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783494    5664 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783497    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.783500    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783592    5664 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783600    5664 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783609    5664 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783616    5664 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783622    5664 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783627    5664 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783636    5664 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783641    5664 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783644    5664 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783647    5664 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783651    5664 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783654    5664 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783657    5664 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783661    5664 flags.go:64] FLAG: --cgroup-root=""
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783664    5664 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783667    5664 flags.go:64] FLAG: --client-ca-file=""
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783669    5664 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 22:19:29.789543 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783672    5664 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783677    5664 flags.go:64] FLAG: --cluster-domain=""
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783680    5664 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783683    5664 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783686    5664 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783690    5664 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783693    5664 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783697    5664 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783700    5664 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783703    5664 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783706    5664 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783710    5664 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783712    5664 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783716    5664 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783718    5664 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783723    5664 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783726    5664 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783729    5664 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783731    5664 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783734    5664 flags.go:64] FLAG: --enable-server="true"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783737    5664 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783758    5664 flags.go:64] FLAG: --event-burst="100"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783763    5664 flags.go:64] FLAG: --event-qps="50"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783766    5664 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 22:19:29.790372 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783770    5664 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783773    5664 flags.go:64] FLAG: --eviction-hard=""
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783778    5664 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783781    5664 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783784    5664 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783787    5664 flags.go:64] FLAG: --eviction-soft=""
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783790    5664 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783793    5664 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783796    5664 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783799    5664 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783802    5664 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783805    5664 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783808    5664 flags.go:64] FLAG: --feature-gates=""
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783812    5664 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783815    5664 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783818    5664 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783822    5664 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783825    5664 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783828    5664 flags.go:64] FLAG: --help="false"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783831    5664 flags.go:64] FLAG: --hostname-override="ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783834    5664 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783837    5664 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783841    5664 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 22:19:29.791359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783844    5664 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783847    5664 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783850    5664 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783853    5664 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783856    5664 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783859    5664 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783862    5664 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783865    5664 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783868    5664 flags.go:64] FLAG: --kube-reserved=""
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783871    5664 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783873    5664 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783877    5664 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783879    5664 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783882    5664 flags.go:64] FLAG: --lock-file=""
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783885    5664 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783888    5664 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783891    5664 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783909    5664 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783913    5664 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783916    5664 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783918    5664 flags.go:64] FLAG: --logging-format="text"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783921    5664 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783925    5664 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783927    5664 flags.go:64] FLAG: --manifest-url=""
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783931    5664 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 22:19:29.792070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783935    5664 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783939    5664 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783943    5664 flags.go:64] FLAG: --max-pods="110"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783946    5664 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783949    5664 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783952    5664 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783955    5664 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783958    5664 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783961    5664 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783964    5664 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783972    5664 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783975    5664 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783978    5664 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783981    5664 flags.go:64] FLAG: --pod-cidr=""
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783983    5664 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783986    5664 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783989    5664 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783992    5664 flags.go:64] FLAG: --port="10250"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783995    5664 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.783998    5664 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0681f0f0ace93dac9"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784001    5664 flags.go:64] FLAG: --qos-reserved=""
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784004    5664 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784007    5664 flags.go:64] FLAG: --register-node="true"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784012    5664 flags.go:64] FLAG: --register-with-taints=""
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784015    5664 flags.go:64] FLAG: --registry-burst="10"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784018    5664 flags.go:64] FLAG: --registry-qps="5"
Oct 30 22:19:29.792697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784021    5664 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784024    5664 flags.go:64] FLAG: --reserved-memory=""
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784028    5664 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784031    5664 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784035    5664 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784038    5664 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784042    5664 flags.go:64] FLAG: --runonce="false"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784046    5664 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784049    5664 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784052    5664 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784055    5664 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784058    5664 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784061    5664 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784064    5664 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784067    5664 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784071    5664 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784074    5664 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784077    5664 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784080    5664 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784083    5664 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784086    5664 flags.go:64] FLAG: --system-cgroups=""
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784088    5664 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784094    5664 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784097    5664 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784100    5664 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 22:19:29.793474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784104    5664 flags.go:64] FLAG: --tls-min-version=""
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784106    5664 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784109    5664 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784112    5664 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784116    5664 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784119    5664 flags.go:64] FLAG: --v="2"
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784123    5664 flags.go:64] FLAG: --version="false"
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784128    5664 flags.go:64] FLAG: --vmodule=""
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784132    5664 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784135    5664 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784285    5664 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784291    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784294    5664 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784297    5664 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784300    5664 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784303    5664 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784307    5664 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784310    5664 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784313    5664 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784315    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784318    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784320    5664 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:29.794176 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784323    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784325    5664 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784328    5664 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784331    5664 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784334    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784337    5664 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784339    5664 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784342    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784345    5664 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784347    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784350    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784352    5664 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784355    5664 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784358    5664 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784362    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784365    5664 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784367    5664 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784370    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784373    5664 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:29.794728 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784375    5664 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784378    5664 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784382    5664 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784384    5664 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784387    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784389    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784392    5664 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784394    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784397    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784400    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784404    5664 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784407    5664 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784410    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784413    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784416    5664 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784419    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784421    5664 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784424    5664 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784426    5664 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:29.795412 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784428    5664 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784431    5664 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784434    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784436    5664 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784439    5664 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784441    5664 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784444    5664 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784447    5664 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784451    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784453    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784456    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784458    5664 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784461    5664 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784463    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784466    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784470    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784473    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784476    5664 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784478    5664 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784481    5664 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:29.796039 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784483    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784486    5664 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784490    5664 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784492    5664 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784495    5664 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784499    5664 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784503    5664 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784506    5664 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784509    5664 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784511    5664 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784513    5664 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784516    5664 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784519    5664 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784521    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784524    5664 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784526    5664 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784529    5664 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784531    5664 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784534    5664 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784536    5664 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:29.796628 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784541    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784543    5664 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784546    5664 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784548    5664 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784551    5664 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784553    5664 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784556    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784560    5664 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784563    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784566    5664 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784568    5664 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784571    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784573    5664 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784576    5664 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784578    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784581    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:29.797494 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.784607    5664 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784644    5664 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784648    5664 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784651    5664 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784653    5664 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784657    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784659    5664 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784662    5664 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784665    5664 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784667    5664 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784670    5664 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784672    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784675    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784677    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784680    5664 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784682    5664 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784686    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784689    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784691    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784694    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:29.797923 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784696    5664 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784699    5664 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784702    5664 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784704    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784708    5664 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784710    5664 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784713    5664 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784715    5664 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784720    5664 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784724    5664 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784727    5664 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784730    5664 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784734    5664 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784736    5664 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784739    5664 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784742    5664 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784744    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784747    5664 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784749    5664 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784751    5664 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:29.798432 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784754    5664 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784756    5664 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784759    5664 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784761    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784764    5664 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784766    5664 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784769    5664 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784771    5664 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784775    5664 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784778    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784781    5664 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784783    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784786    5664 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784789    5664 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784791    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784793    5664 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784798    5664 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784800    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784803    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:29.798994 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784805    5664 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784808    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784810    5664 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784813    5664 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784815    5664 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784818    5664 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784821    5664 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784824    5664 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784827    5664 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784829    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784832    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784834    5664 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784837    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784839    5664 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784842    5664 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784844    5664 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784847    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784849    5664 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784852    5664 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784854    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:29.799579 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784857    5664 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784861    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784864    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784866    5664 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784869    5664 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784872    5664 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784874    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784877    5664 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784879    5664 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784883    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784885    5664 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784888    5664 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784891    5664 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784906    5664 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784909    5664 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784911    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784914    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784917    5664 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784919    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:29.800180 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784922    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784925    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784930    5664 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784933    5664 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784936    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784939    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784942    5664 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784945    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.784948    5664 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.789176    5664 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.789190    5664 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.789210    5664 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.789215    5664 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789283    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789289    5664 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789292    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789295    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789298    5664 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789301    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:29.801098 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789304    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789307    5664 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789309    5664 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789312    5664 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789315    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789317    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789320    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789323    5664 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789325    5664 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789328    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789331    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789333    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789335    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789344    5664 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789347    5664 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789350    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789353    5664 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789355    5664 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789358    5664 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789361    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:29.801888 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789363    5664 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789366    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789369    5664 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789371    5664 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789373    5664 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789376    5664 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789378    5664 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789381    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789384    5664 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789386    5664 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789389    5664 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789392    5664 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789395    5664 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789398    5664 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789400    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789403    5664 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789405    5664 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789408    5664 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789410    5664 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789413    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:29.802743 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789416    5664 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789418    5664 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789421    5664 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789423    5664 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789426    5664 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789429    5664 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789437    5664 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789440    5664 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789443    5664 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789445    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789448    5664 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789450    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789453    5664 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789455    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789458    5664 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789461    5664 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789463    5664 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789466    5664 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789469    5664 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:29.803629 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789473    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789475    5664 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789478    5664 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789481    5664 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789483    5664 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789486    5664 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789488    5664 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789491    5664 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789493    5664 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789496    5664 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789498    5664 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789501    5664 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789503    5664 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789506    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789508    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789511    5664 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789513    5664 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789516    5664 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789519    5664 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:29.804449 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789521    5664 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789530    5664 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789533    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789537    5664 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789540    5664 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789543    5664 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789546    5664 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789548    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789551    5664 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789554    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789556    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789559    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789561    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789564    5664 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789566    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789569    5664 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789571    5664 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789574    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789576    5664 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:29.805283 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789579    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789581    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.789584    5664 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.789611    5664 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790516    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790706    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790712    5664 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790723    5664 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790729    5664 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790733    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790737    5664 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790741    5664 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790746    5664 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790750    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790754    5664 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790789    5664 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:29.806134 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790794    5664 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790799    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790803    5664 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790812    5664 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790817    5664 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790821    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790825    5664 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790829    5664 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790833    5664 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790838    5664 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790843    5664 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790847    5664 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790851    5664 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790855    5664 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790859    5664 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790863    5664 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790872    5664 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790876    5664 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790880    5664 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790884    5664 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:29.806834 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790887    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790891    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790916    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790920    5664 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790924    5664 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790928    5664 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790932    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790936    5664 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790941    5664 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790949    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790953    5664 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790957    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790994    5664 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.790999    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791003    5664 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791007    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791011    5664 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791015    5664 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791019    5664 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791023    5664 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:29.807695 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791028    5664 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791033    5664 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791042    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791047    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791053    5664 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791057    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791062    5664 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791125    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791146    5664 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791153    5664 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791157    5664 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791163    5664 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791167    5664 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791171    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791176    5664 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791180    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791185    5664 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791189    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791194    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:29.808585 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791204    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791212    5664 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791219    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791239    5664 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791242    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791245    5664 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791262    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791396    5664 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791404    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791406    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791409    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791412    5664 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791415    5664 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791420    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791423    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791426    5664 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791429    5664 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791431    5664 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791434    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:29.809444 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791437    5664 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791440    5664 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791442    5664 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791445    5664 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791448    5664 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791450    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791453    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791457    5664 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791462    5664 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791465    5664 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791468    5664 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791471    5664 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791474    5664 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791477    5664 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791479    5664 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:29.810356 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791482    5664 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.791519    5664 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791562    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791567    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791571    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791574    5664 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791578    5664 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791581    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791584    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791587    5664 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791590    5664 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791592    5664 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791599    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791602    5664 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791605    5664 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791608    5664 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 22:19:29.811132 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791611    5664 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791614    5664 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791616    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791619    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791623    5664 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791627    5664 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791630    5664 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791633    5664 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791636    5664 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791638    5664 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791641    5664 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791644    5664 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791647    5664 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791649    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791652    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791654    5664 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791657    5664 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791660    5664 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791662    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 22:19:29.811847 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791665    5664 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791667    5664 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791670    5664 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791673    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791676    5664 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791678    5664 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791680    5664 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791684    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791686    5664 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791690    5664 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791692    5664 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791695    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791698    5664 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791700    5664 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791703    5664 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791705    5664 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791708    5664 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791710    5664 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791712    5664 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791715    5664 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 22:19:29.812856 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791719    5664 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791722    5664 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791724    5664 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791727    5664 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791729    5664 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791732    5664 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791734    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791737    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791739    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791741    5664 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791744    5664 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791746    5664 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791749    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791751    5664 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791753    5664 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791756    5664 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791758    5664 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791761    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791763    5664 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791766    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 22:19:29.813841 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791768    5664 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791772    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791777    5664 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791779    5664 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791782    5664 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791784    5664 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791787    5664 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791790    5664 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791792    5664 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791795    5664 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791797    5664 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791800    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791803    5664 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791806    5664 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791809    5664 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791811    5664 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791814    5664 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791817    5664 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791819    5664 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791822    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 22:19:29.814814 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791825    5664 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791827    5664 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791830    5664 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791832    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791835    5664 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791837    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791842    5664 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791846    5664 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791849    5664 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791852    5664 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791855    5664 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791858    5664 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:29.791860    5664 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.791958    5664 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.792950    5664 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.793006    5664 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.793181    5664 server.go:1014] "Starting client certificate rotation"
Oct 30 22:19:29.815572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.793320    5664 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 22:19:29.816021 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.793374    5664 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 22:13:18 +0000 UTC" deadline="2028-05-08 12:07:56.255913366 +0000 UTC"
Oct 30 22:19:29.816021 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.793401    5664 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13333h48m26.462514584s"
Oct 30 22:19:29.816021 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.793670    5664 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:19:29.816021 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.794813    5664 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 22:19:29.816021 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.796645    5664 log.go:25] "Validated CRI v1 runtime API"
Oct 30 22:19:29.816021 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.798801    5664 log.go:25] "Validated CRI v1 image API"
Oct 30 22:19:29.816021 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.799516    5664 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 22:19:29.816021 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.802737    5664 fs.go:135] Filesystem UUIDs: map[0903cb39-a7be-41be-93ed-3493de8db811:/dev/nvme0n1p3 1facfe9a-58f9-4bca-b5ae-2330f8940992:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Oct 30 22:19:29.816429 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.802766    5664 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/13fbf1123d6b450a07f082ff67b0543ec582413e69ce9216baaaa4c443160063/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/13fbf1123d6b450a07f082ff67b0543ec582413e69ce9216baaaa4c443160063/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2198be6303a3f26b23867e49661a3fb6af743c5d8799fed8d13a7bad12d4a3cb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2198be6303a3f26b23867e49661a3fb6af743c5d8799fed8d13a7bad12d4a3cb/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2279054b88b2fe10242e46ba096b251fcb0dffc9fc308fae535ccfac8053dd17/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2279054b88b2fe10242e46ba096b251fcb0dffc9fc308fae535ccfac8053dd17/userdata/shm major:0 minor:421 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2d0e3b1f8b8dda90a7ff6b39e26ad3dcbbe6661ddbd61e1642bde37b87313fe6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2d0e3b1f8b8dda90a7ff6b39e26ad3dcbbe6661ddbd61e1642bde37b87313fe6/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3a3dec3c50c3664399b01d817628bf0151f02624a2be2a70e3905315c1338337/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3a3dec3c50c3664399b01d817628bf0151f02624a2be2a70e3905315c1338337/userdata/shm major:0 minor:288 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3e06b684166b1b72ff4fb79794ee5176b7133131d9d022fd8ebd1e287d7c2c59/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3e06b684166b1b72ff4fb79794ee5176b7133131d9d022fd8ebd1e287d7c2c59/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4f7a9b4759fb18f1b558d4f329f9ee200a2d4ce5526c9f4d96055dc85a45ff3d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4f7a9b4759fb18f1b558d4f329f9ee200a2d4ce5526c9f4d96055dc85a45ff3d/userdata/shm major:0 minor:423 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5134b33f46911bea4675ef15db577a65f096367fe6edd4731de5e70512bf918e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5134b33f46911bea4675ef15db577a65f096367fe6edd4731de5e70512bf918e/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/57e264ca0570959ff62694180f54ea975cfef446444aab0a3e139e9a48e7faec/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/57e264ca0570959ff62694180f54ea975cfef446444aab0a3e139e9a48e7faec/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/592ebcb1e93b72d18af501402a503e32f6ec8bbf5c1019f4c15ba7b60040355b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/592ebcb1e93b72d18af501402a503e32f6ec8bbf5c1019f4c15ba7b60040355b/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7539f30ff407b8642f151d28b324a4fb27c751a44529fb60cc780195b0e4d274/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7539f30ff407b8642f151d28b324a4fb27c751a44529fb60cc780195b0e4d274/userdata/shm major:0 minor:378 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7821eabf282976a84916b1655ea63d0434909eecbf1475fcee413a7057f48df3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7821eabf282976a84916b1655ea63d0434909eecbf1475fcee413a7057f48df3/userdata/shm major:0 minor:425 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8d23da89396e0eb3763a357b85fe9e4bb4cdae904ab8265915f2ecb014d79cd0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8d23da89396e0eb3763a357b85fe9e4bb4cdae904ab8265915f2ecb014d79cd0/userdata/shm major:0 minor:237 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/99392956d77083b97274fcae86e3dba3c838d69117f3d473c2c3d6f12573bf49/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/99392956d77083b97274fcae86e3dba3c838d69117f3d473c2c3d6f12573bf49/userdata/shm major:0 minor:243 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/af40adc363aca75eba397b156ee99c197cf1f27cfc50eb96a17acf47b6e1541a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/af40adc363aca75eba397b156ee99c197cf1f27cfc50eb96a17acf47b6e1541a/userdata/shm major:0 minor:87 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c134e4cc8deb953654e6b44f4087954e8815bd9e8a6aed22738757ff72a2f018/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c134e4cc8deb953654e6b44f4087954e8815bd9e8a6aed22738757ff72a2f018/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c2f2693b5f0b2da6f487519177d18b9e4a12b66c5e724db5ee720653b48698c8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c2f2693b5f0b2da6f487519177d18b9e4a12b66c5e724db5ee720653b48698c8/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/caf23f7035b5d40672afc6969478cb1818857d9a1c0efb3528a36b85b488798a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/caf23f7035b5d40672afc6969478cb1818857d9a1c0efb3528a36b85b488798a/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ddd09928a7a0cd62a016a44fcc552e3de824cf3a711ed70bddf0e9b82e669f35/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ddd09928a7a0cd62a016a44fcc552e3de824cf3a711ed70bddf0e9b82e669f35/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e23ef180860396c0b4899f2c6a7b424655b12cf8f073880fd8d2e5083cf8a563/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e23ef180860396c0b4899f2c6a7b424655b12cf8f073880fd8d2e5083cf8a563/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fba94a278e13d0b321e64c666787eb0e317aed77974fbf8ff88d74b7dda6ba24/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fba94a278e13d0b321e64c666787eb0e317aed77974fbf8ff88d74b7dda6ba24/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fcdb921f5a039678d98a0e65e7a92593630132e11262305e0b58767321407e56/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fcdb921f5a039678d98a0e65e7a92593630132e11262305e0b58767321407e56/userdata/shm major:0 minor:263 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/web-config/alertmanager/9 major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/web-config/config-reloader/6 major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~empty-dir/config-out major:0 minor:285 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~projected/kube-api-access-z2vgr:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~projected/kube-api-access-z2vgr major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~projected/tls-assets major:0 minor:282 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:280 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/config-volume major:0 minor:283 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:281 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:279 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/web-config major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a882ca4-1b11-494c-a51e-fb3c93ef5606/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/1a882ca4-1b11-494c-a51e-fb3c93ef5606/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:415 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6/volumes/kubernetes.io~projected/kube-api-access-6jbvf:{mountpoint:/var/lib/kubelet/pods/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6/volumes/kubernetes.io~projected/kube-api-access-6jbvf major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2fb7c085-6b91-409e-b15b-6d5b94f1c94c/volumes/kubernetes.io~projected/kube-api-access-xw7st:{mountpoint:/var/lib/kubelet/pods/2fb7c085-6b91-409e-b15b-6d5b94f1c94c/volumes/kubernetes.io~projected/kube-api-access-xw7st major:0 minor:234 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2fb7c085-6b91-409e-b15b-6d5b94f1c94c/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/2fb7c085-6b91-409e-b15b-6d5b94f1c94c/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2fb7c085-6b91-409e-b15b-6d5b94f1c94c/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/2fb7c085-6b91-409e-b15b-6d5b94f1c94c/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:231 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04/volumes/kubernetes.io~projected/kube-api-access-xrdzx:{mountpoint:/var/lib/kubelet/pods/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04/volumes/kubernetes.io~projected/kube-api-access-xrdzx major:0 minor:420 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/41e14309-abd7-407f-b98b-39005ec7d563/volumes/kubernetes.io~projected/kube-api-access-lqmvd:{mountpoint:/var/lib/kubelet/pods/41e14309-abd7-407f-b98b-39005ec7d563/volumes/kubernetes.io~projected/kube-api-access-lqmvd major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60e9435e-1582-493b-a73d-226b32c86abc/volumes/kubernetes.io~projected/kube-api-access-4m8hc:{mountpoint:/var/lib/kubelet/pods/60e9435e-1582-493b-a73d-226b32c86abc/volumes/kubernetes.io~projected/kube-api-access-4m8hc major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/60e9435e-1582-493b-a73d-226b32c86abc/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/60e9435e-1582-493b-a73d-226b32c86abc/volumes/kubernetes.io~secret/cert major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7110fffb-3090-497a-a6d8-0260f3166ed1/volumes/kubernetes.io~projected/kube-api-access-kb2jv:{mountpoint:/var/lib/kubelet/pods/7110fffb-3090-497a-a6d8-0260f3166ed1/volumes/kubernetes.io~projected/kube-api-access-kb2jv major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7110fffb-3090-497a-a6d8-0260f3166ed1/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/7110fffb-3090-497a-a6d8-0260f3166ed1/volumes/kubernetes.io~secret/metrics-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/72a3a151-21d2-4551-94b5-332f0b4f9542/volumes/kubernetes.io~projected/kube-api-access-cjdcx:{mountpoint:/var/lib/kubelet/pods/72a3a151-21d2-4551-94b5-332f0b4f9542/volumes/kubernetes.io~projected/kube-api-access-cjdcx major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76ab3954-a12f-484a-b32f-3de7517bc70f/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/76ab3954-a12f-484a-b32f-3de7517bc70f/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76ab3954-a12f-484a-b32f-3de7517bc70f/volumes/kubernetes.io~projected/kube-api-access-l9prx:{mountpoint:/var/lib/kubelet/pods/76ab3954-a12f-484a-b32f-3de7517bc70f/volumes/kubernetes.io~projected/kube-api-access-l9prx major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/76ab3954-a12f-484a-b32f-3de7517bc70f/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/76ab3954-a12f-484a-b32f-3de7517bc70f/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77422c7f-1d2c-4ca8-968c-3e59beffbb1e/volumes/kubernetes.io~projected/kube-api-access-mvw67:{mountpoint:/var/lib/kubelet/pods/77422c7f-1d2c-4ca8-968c-3e59beffbb1e/volumes/kubernetes.io~projected/kube-api-access-mvw67 major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77422c7f-1d2c-4ca8-968c-3e59beffbb1e/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/77422c7f-1d2c-4ca8-968c-3e59beffbb1e/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77422c7f-1d2c-4ca8-968c-3e59beffbb1e/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/77422c7f-1d2c-4ca8-968c-3e59beffbb1e/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:230 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/777ef147-4f3f-4c89-ae2b-545aad6b31f1/volumes/kubernetes.io~projected/kube-api-access-dlz2q:{mountpoint:/var/lib/kubelet/pods/777ef147-4f3f-4c89-ae2b-545aad6b31f1/volumes/kubernetes.io~projected/kube-api-access-dlz2q major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa/volumes/kubernetes.io~projected/kube-api-access-lf9fr:{mountpoint:/var/lib/kubelet/pods/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa/volumes/kubernetes.io~projected/kube-api-access-lf9fr major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa/volumes/kubernetes.io~secret/metrics-certs major:0 minor:419 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/volumes/kubernetes.io~projected/kube-api-access-gng5d:{mountpoint:/var/lib/kubelet/pods/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/volumes/kubernetes.io~projected/kube-api-access-gng5d major:0 minor:235 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:233 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb6f24bb-ba37-400e-878f-267797c1903b/volumes/kubernetes.io~projected/kube-api-access-6xtbl:{mountpoint:/var/lib/kubelet/pods/bb6f24bb-ba37-400e-878f-267797c1903b/volumes/kubernetes.io~projected/kube-api-access-6xtbl major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes/kubernetes.io~projected/kube-api-access-f78rc:{mountpoint:/var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes/kubernetes.io~projected/kube-api-access-f78rc major:0 minor:271 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d9093009-6c67-4d6b-8d3a-a388dbb078a3/volumes/kubernetes.io~projected/kube-api-access-ldx4p:{mountpoint:/var/lib/kubelet/pods/d9093009-6c67-4d6b-8d3a-a388dbb078a3/volumes/kubernetes.io~projected/kube-api-access-ldx4p major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/volumes/kubernetes.io~projected/kube-api-access-h824h:{mountpoint:/var/lib/kubelet/pods/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/volumes/kubernetes.io~projected/kube-api-access-h824h major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volume-subpaths/web-config/config-reloader/2:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volume-subpaths/web-config/config-reloader/2 major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volume-subpaths/web-config/init-config-reloader/2:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volume-subpaths/web-config/init-config-reloader/2 major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volume-subpaths/web-config/prometheus/15:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volume-subpaths/web-config/prometheus/15 major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~empty-dir/config-out major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~projected/kube-api-access-pnlcg:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~projected/kube-api-access-pnlcg major:0 minor:377 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~projected/tls-assets major:0 minor:373 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/config:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/config major:0 minor:375 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:369 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-kube-rbac-proxy major:0 minor:371 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-metrics-client-certs:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-metrics-client-certs major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web major:0 minor:374 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls major:0 minor:372 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls major:0 minor:376 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file major:0 minor:370 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/web-config major:0 minor:368 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f6695e08-7390-47f0-b5db-f474a466cf0a/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/f6695e08-7390-47f0-b5db-f474a466cf0a/volumes/kubernetes.io~secret/agent-certs major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f9eaa777-0ec3-4c87-92a6-05e46f50066f/volumes/kubernetes.io~projected/kube-api-access-nspxc:{mountpoint:/var/lib/kubelet/pods/f9eaa777-0ec3-4c87-92a6-05e46f50066f/volumes/kubernetes.io~projected/kube-api-access-nspxc major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f9eaa777-0ec3-4c87-92a6-05e46f50066f/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/f9eaa777-0ec3-4c87-92a6-05e46f50066f/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/7d0c10d77b81f61ff6260dd5e78326915c46245e287a600b04551e2ffb3d45fb/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/01429e03fba57f0dfb49c21a74cf1895b5bd25eae67395160b308e519c358a2d/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/4db2de84c0c3e9baed78d41e37add4bc2f236d6e810b2c898e7aac3348ccc89e/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/893f9de3c7e157e9cfd77f8ef674d0a0afe71ecd0665dd3f89abc1d53becfd42/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/328c1652f14366615dd60d65b83557dbe3afcee72d85da143a80fff34de7dd4b/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/17af551fd87dd4d1ecc0873bf7f9f5b1f5225bfd539c80f400b8cbe2c89a21df/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/41f1c7feb091fa55f2b0714efe472ddec1faf874c7d2ceee934e5510248773b8/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/89ab91f149b356ff0f23d373f24a89ba613e098a555fbe051412458f4991daee/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/883d0f90f7d1450a9951e6e02cd8b8b95e20ecb68bbe0e557b9de060ab898e65/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/e5c5bbb2b4dabe3a04533e3df8e2f9bace2072724a9fc7a3db4b22f4e6810012/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-141:{mountpoint:/var/lib/containers/storage/overlay/77bbe5095a461093bd38cd6fc24d2cb90411d0a529fa5311ae09e44b6133e5b6/merged major:0 minor:141 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/0b215ee7aba115b87447580a269e5c0597cd71055c79b893789735fff2971bed/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/2c527633b7bac03414b80dccc59d70487d309d63cc70316fe62ba415a1305e56/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-158:{mountpoint:/var/lib/containers/storage/overlay/f0112caa5b2e5b479a035ab56f31fe2d4dce6f3c307aac12164b4b9eec9a610c/merged major:0 minor:158 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/0ba9120cec5d77904888247c054c20ebcd7c399e97504f433f194bc5244294ac/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/cca479ef67b15acbc64b42f6538ce21b0e639f83035cab48b7b539eb7fcfc062/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/9f0e5a992c1fba24258edbca8b6ac8b857df0d9fd0cefda31e2d60682b0e44d7/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/39bb745f6a86cb87c37efb99aa8f7a40868e3f4424452296460c7da70a24014c/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/b106258e239047132bc4dfb1792db338db0a711aa7db018d3896660a94e33724/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/4da8d8fa77ab2d7b149db23e400080a76d597a81826e8a22ef4bcbc7d2c677b5/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/edd51d169e02d9038c8bdc8d426fd78203f6a4adc55b9b70eab1dcd00dc7e447/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/b2e6da861b26dad15377edc3ae400703e44118246ac6eb077365099cca2b481a/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/9004665b68d5894cc922f7aeed1854302c0376463b606185842a3060a94d31a7/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/6e4161b38592b4e85238f7e35f5b9db9f779d38496f42ee62b7987eda9a97e87/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/d4078aa0f1205ec54fc22e82e4822d11821577a8e42bb1c14186eb30be01ed34/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/5f50e959e694835b62b45690560617e224c738ba93d1686393a737fd5846425b/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/0bd45bae0a7490eeffe62a9694645535bea5ec85371d841b6c763bec3245fb4b/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-245:{mountpoint:/var/lib/containers/storage/overlay/579d761fb67f437eee2b830aae36c0605eaf5412833977ec72782eca454501e5/merged major:0 minor:245 fsType:overlay blockSize:0} overlay_0-247:{mountpoint:/var/lib/containers/storage/overlay/c6bfd09a63f37d44cfa6e731ef2bb779ffdba0fd5e8c2dc9f66cb7abc1b27ee2/merged major:0 minor:247 fsType:overlay blockSize:0} overlay_0-249:{mountpoint:/var/lib/containers/storage/overlay/46fbaa5377c387e8b305a5b7af8758be61c4ded986f31fa65bc69cd62e6bd1fd/merged major:0 minor:249 fsType:overlay blockSize:0} overlay_0-251:{mountpoint:/var/lib/containers/storage/overlay/78d732689fb5425e17e45c40aad9bf3014ddbb0e03873c8dbb8c7b8aecdf10c5/merged major:0 minor:251 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/7526ac542e07e51037951cef7b7216aa00e640d6534acf22a326c6acfba5b081/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/c15b276cdd84de45b95b42c8c318628ae6db55a3dbc8326e1becdd04113d561f/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-286:{mountpoint:/var/lib/containers/storage/overlay/fddbdbf62582222db184d441d33d6eee60d76e89998e138119936f72e5977dcb/merged major:0 minor:286 fsType:overlay blockSize:0} overlay_0-290:{mountpoint:/var/lib/containers/storage/overlay/660ab1dc0ee607f214d4ef4384fc4e1607e3a14b9c98016dcaee40f5f9648ec4/merged major:0 minor:290 fsType:overlay blockSize:0} overlay_0-292:{mountpoint:/var/lib/containers/storage/overlay/57b56d13990ad72a5bccea582b819fc4fc668eec1e486dcad81f1ae530437d5e/merged major:0 minor:292 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/3e3773fbf629a0035c716b8ac91b1c8d77264236ff1a6783e953dbe4d080461d/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/9d9dc237cf33771960e419096c1d8134abed999f1baac38481b1514d8647b737/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/fea06048ddb96ceb05708958ca9b316ce117c0e951b52d4237271a73ecde9352/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/a28b59a9368d094b3c5864e07db764f70d4a31c1cbf96118139f989813465c25/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/7ad3b49b6d6bf9b8440cb8d51274490288c3c27c93db61b7634d51bf745b069e/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-311:{mountpoint:/var/lib/containers/storage/overlay/a6aa81a4fbabe13ca654f351db85c8d506c360645e315582fc0e6c5ea06561b9/merged major:0 minor:311 fsType:overlay blockSize:0} overlay_0-323:{mountpoint:/var/lib/containers/storage/overlay/ff44fdb71e41703213a6489e741d3bd7c81d0bf4e93259ba60597f89a398ebd9/merged major:0 minor:323 fsType:overlay blockSize:0} overlay_0-331:{mountpoint:/var/lib/containers/storage/overlay/d612c0331f5f34471c76a4db8672dcce6b561a2d4d4b7329e3c2838bfa560021/merged major:0 minor:331 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/6b51ea93973191b510996d1d245afded7cac33d7557f40707bcec5ed59ad8173/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-342:{mountpoint:/var/lib/containers/storage/overlay/972992572c4ce5a2899c62780488cfbd5704f79898fcceb338bece30eddc16e4/merged major:0 minor:342 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/5df7839c1227511a9d0f6052a398f3fe9a1030aca8b72aca3b480d0e4f0468e4/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-353:{mountpoint:/var/lib/containers/storage/overlay/628d62da44e703583b9b2a4301a11545ab6ee1530697511d82f5fdbffff76efc/merged major:0 minor:353 fsType:overlay blockSize:0} overlay_0-358:{mountpoint:/var/lib/containers/storage/overlay/3cf8b25d9ffa5b36c95147e2b4751c4fa77dd4498eaf3787e9484bb655ad081d/merged major:0 minor:358 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/61b0866565507d09c8b39491e704a6376331655004d1f258386634b39f174513/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-382:{mountpoint:/var/lib/containers/storage/overlay/ddd36025c2c23aa2107c2178023f4043148774c614e2865e1a105bb1f891eb51/merged major:0 minor:382 fsType:overlay blockSize:0} overlay_0-384:{mountpoint:/var/lib/containers/storage/overlay/af6701ccce76124ab2df7304323ead007f40144dbd288a5065ce0ccab3fae715/merged major:0 minor:384 fsType:overlay blockSize:0} overlay_0-389:{mountpoint:/var/lib/containers/storage/overlay/455a0cf092b22213bfad03936005a50a1ad26a285e57de7b07eaf7ee88819caf/merged major:0 minor:389 fsType:overlay blockSize:0} overlay_0-395:{mountpoint:/var/lib/containers/storage/overlay/d6c39c84f41d55947eb1e38a8b86d19ee7da4ea98b6f7e760e979f193d7ad3e0/merged major:0 minor:395 fsType:overlay blockSize:0} overlay_0-400:{mountpoint:/var/lib/containers/storage/overlay/b874792436f01aff7aa1e17e070deca7b9d44caa213f2230945313f2c6286755/merged major:0 minor:400 fsType:overlay blockSize:0} overlay_0-405:{mountpoint:/var/lib/containers/storage/overlay/c09591ff092c464dce19dc0e3ce2fea6851fb8178263b601766266b9f3695813/merged major:0 minor:405 fsType:overlay blockSize:0} overlay_0-410:{mountpoint:/var/lib/containers/storage/overlay/731378ec9ad8cdd163ca41bb6f7399c970640e9b9e40b12add2d9bef01655e37/merged major:0 minor:410 fsType:overlay blockSize:0} overlay_0-427:{mountpoint:/var/lib/containers/storage/overlay/6cb25a9c0d66c620fe5ca4def05e175ea8660b79c3c05c7f87cbde2dd220b0e7/merged major:0 minor:427 fsType:overlay blockSize:0} overlay_0-429:{mountpoint:/var/lib/containers/storage/overlay/fd02ae8610f75b4953a7d361df98f74a607be217aa83667a77c74fb05eac023c/merged major:0 minor:429 fsType:overlay blockSize:0} overlay_0-431:{mountpoint:/var/lib/containers/storage/overlay/f7bda33179a8e9be91a853403ee05feff0161673163b89c66a74fb01ae3ade5c/merged major:0 minor:431 fsType:overlay blockSize:0} overlay_0-433:{mountpoint:/var/lib/containers/storage/overlay/6117f6de264dd931986c00a54386add0c50771cc2b8496607a89fb9bc8643966/merged major:0 minor:433 fsType:overlay blockSize:0} overlay_0-435:{mountpoint:/var/lib/containers/storage/overlay/e1d69bbfbec51f8c033492e68933d2b29b6276e865f80f9f250f1f8933d69743/merged major:0 minor:435 fsType:overlay blockSize:0} overlay_0-441:{mountpoint:/var/lib/containers/storage/overlay/1fe5959d62ff0c01654ee6b948e6d756da15ab0b75499c23f1e1edb28a980b16/merged major:0 minor:441 fsType:overlay blockSize:0} overlay_0-446:{mountpoint:/var/lib/containers/storage/overlay/5be0517fb2312648c678a54c558a8082b418e43523c5d083509a2a591fbe38ca/merged major:0 minor:446 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/ea12c5a9866206eafdc47d4ed8d5ea5f6d48e8ecdf4c2f24b01daa7560039f36/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/dac53ab08e5c40ab0e55b9554e6584ba1e32d55cd947dc6a3a1ff722289b59f5/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/b87d233bdf4c046034ab6918480d2389656c46ebee37a3aa995d866769f37d3d/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/b6515073c64f0948a707df788a1dec033cf94d90d0daef6c60766593f573b04e/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/53df16eae8e10ec9c8fc1ffb993d862bb6a1b4a151938c437d5a9deb8955cf2f/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/cc5bceb937dd48209a4f4308786842cbd6262e50926e413c166349892f68d76e/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/5b2abd7ea13d63594e6669e0764b2840928bd772aefdfd070fb205ac5f6a6a3b/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-89:{mountpoint:/var/lib/containers/storage/overlay/c597382401924875b92c4ed199b05503a1a72811b91dcf4e812128a841c93c1c/merged major:0 minor:89 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/7af704f392a0b37996c25ef0a23a022a31b9e8244c2c1733f2dc4dd8cd6721c3/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/37aa789bc4cb14d605f70693329691a99a50b9da9b9051cbb9772ab29c134f27/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.815462    5664 manager.go:217] Machine: {Timestamp:2026-10-30 22:19:29.814649302 +0000 UTC m=+0.066112762 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099271 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e8f28d9cbbde52f989bb6ff709893 SystemUUID:ec2e8f28-d9cb-bde5-2f98-9bb6ff709893 BootID:5c1b667f-1918-45df-8038-4a4f26a8bcbd Filesystems:[{Device:/run/containers/storage/overlay-containers/ddd09928a7a0cd62a016a44fcc552e3de824cf3a711ed70bddf0e9b82e669f35/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f9eaa777-0ec3-4c87-92a6-05e46f50066f/volumes/kubernetes.io~projected/kube-api-access-nspxc DeviceMajor:0 DeviceMinor:202 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-251 DeviceMajor:0 DeviceMinor:251 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:279 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~projected/kube-api-access-z2vgr DeviceMajor:0 DeviceMinor:284 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:285 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/thanos-prometheus-http-client-file DeviceMajor:0 DeviceMinor:370 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/13fbf1123d6b450a07f082ff67b0543ec582413e69ce9216baaaa4c443160063/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2279054b88b2fe10242e46ba096b251fcb0dffc9fc308fae535ccfac8053dd17/userdata/shm DeviceMajor:0 DeviceMinor:421 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-431 DeviceMajor:0 DeviceMinor:431 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/volumes/kubernetes.io~projected/kube-api-access-h824h DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/41e14309-abd7-407f-b98b-39005ec7d563/volumes/kubernetes.io~projected/kube-api-access-lqmvd DeviceMajor:0 DeviceMinor:63 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77422c7f-1d2c-4ca8-968c-3e59beffbb1e/volumes/kubernetes.io~projected/kube-api-access-mvw67 DeviceMajor:0 DeviceMinor:236 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3a3dec3c50c3664399b01d817628bf0151f02624a2be2a70e3905315c1338337/userdata/shm DeviceMajor:0 DeviceMinor:288 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-290 DeviceMajor:0 DeviceMinor:290 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:277 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-358 DeviceMajor:0 DeviceMinor:358 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7110fffb-3090-497a-a6d8-0260f3166ed1/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/volumes/kubernetes.io~projected/kube-api-access-gng5d DeviceMajor:0 DeviceMinor:235 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-292 DeviceMajor:0 DeviceMinor:292 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-prometheus-k8s-thanos-sidecar-tls DeviceMajor:0 DeviceMinor:372 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/76ab3954-a12f-484a-b32f-3de7517bc70f/volumes/kubernetes.io~projected/kube-api-access-l9prx DeviceMajor:0 DeviceMinor:66 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3e06b684166b1b72ff4fb79794ee5176b7133131d9d022fd8ebd1e287d7c2c59/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-342 DeviceMajor:0 DeviceMinor:342 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volume-subpaths/web-config/init-config-reloader/2 DeviceMajor:0 DeviceMinor:368 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:281 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:280 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/57e264ca0570959ff62694180f54ea975cfef446444aab0a3e139e9a48e7faec/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa/volumes/kubernetes.io~projected/kube-api-access-lf9fr DeviceMajor:0 DeviceMinor:65 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77422c7f-1d2c-4ca8-968c-3e59beffbb1e/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:230 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/72a3a151-21d2-4551-94b5-332f0b4f9542/volumes/kubernetes.io~projected/kube-api-access-cjdcx DeviceMajor:0 DeviceMinor:62 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:278 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-323 DeviceMajor:0 DeviceMinor:323 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:270 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:277 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c2f2693b5f0b2da6f487519177d18b9e4a12b66c5e724db5ee720653b48698c8/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fba94a278e13d0b321e64c666787eb0e317aed77974fbf8ff88d74b7dda6ba24/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:276 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-286 DeviceMajor:0 DeviceMinor:286 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-353 DeviceMajor:0 DeviceMinor:353 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-410 DeviceMajor:0 DeviceMinor:410 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-89 DeviceMajor:0 DeviceMinor:89 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-427 DeviceMajor:0 DeviceMinor:427 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/caf23f7035b5d40672afc6969478cb1818857d9a1c0efb3528a36b85b488798a/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/af40adc363aca75eba397b156ee99c197cf1f27cfc50eb96a17acf47b6e1541a/userdata/shm DeviceMajor:0 DeviceMinor:87 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:233 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fcdb921f5a039678d98a0e65e7a92593630132e11262305e0b58767321407e56/userdata/shm DeviceMajor:0 DeviceMinor:263 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:283 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-prometheus-k8s-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:374 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:269 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-kube-rbac-proxy DeviceMajor:0 DeviceMinor:371 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/config DeviceMajor:0 DeviceMinor:375 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-331 DeviceMajor:0 DeviceMinor:331 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volume-subpaths/web-config/config-reloader/2 DeviceMajor:0 DeviceMinor:368 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f9eaa777-0ec3-4c87-92a6-05e46f50066f/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/60e9435e-1582-493b-a73d-226b32c86abc/volumes/kubernetes.io~projected/kube-api-access-4m8hc DeviceMajor:0 DeviceMinor:203 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04/volumes/kubernetes.io~projected/kube-api-access-xrdzx DeviceMajor:0 DeviceMinor:420 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/777ef147-4f3f-4c89-ae2b-545aad6b31f1/volumes/kubernetes.io~projected/kube-api-access-dlz2q DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2198be6303a3f26b23867e49661a3fb6af743c5d8799fed8d13a7bad12d4a3cb/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-311 DeviceMajor:0 DeviceMinor:311 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6/volumes/kubernetes.io~projected/kube-api-access-6jbvf DeviceMajor:0 DeviceMinor:67 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5134b33f46911bea4675ef15db577a65f096367fe6edd4731de5e70512bf918e/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77422c7f-1d2c-4ca8-968c-3e59beffbb1e/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:232 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-247 DeviceMajor:0 DeviceMinor:247 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-249 DeviceMajor:0 DeviceMinor:249 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-prometheus-k8s-tls DeviceMajor:0 DeviceMinor:376 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/99392956d77083b97274fcae86e3dba3c838d69117f3d473c2c3d6f12573bf49/userdata/shm DeviceMajor:0 DeviceMinor:243 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-245 DeviceMajor:0 DeviceMinor:245 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~projected/kube-api-access-pnlcg DeviceMajor:0 DeviceMinor:377 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4f7a9b4759fb18f1b558d4f329f9ee200a2d4ce5526c9f4d96055dc85a45ff3d/userdata/shm DeviceMajor:0 DeviceMinor:423 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/76ab3954-a12f-484a-b32f-3de7517bc70f/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/60e9435e-1582-493b-a73d-226b32c86abc/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e23ef180860396c0b4899f2c6a7b424655b12cf8f073880fd8d2e5083cf8a563/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:369 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-382 DeviceMajor:0 DeviceMinor:382 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7821eabf282976a84916b1655ea63d0434909eecbf1475fcee413a7057f48df3/userdata/shm DeviceMajor:0 DeviceMinor:425 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f6695e08-7390-47f0-b5db-f474a466cf0a/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7110fffb-3090-497a-a6d8-0260f3166ed1/volumes/kubernetes.io~projected/kube-api-access-kb2jv DeviceMajor:0 DeviceMinor:204 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/secret-metrics-client-certs DeviceMajor:0 DeviceMinor:367 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7539f30ff407b8642f151d28b324a4fb27c751a44529fb60cc780195b0e4d274/userdata/shm DeviceMajor:0 DeviceMinor:378 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-384 DeviceMajor:0 DeviceMinor:384 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-389 DeviceMajor:0 DeviceMinor:389 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-400 DeviceMajor:0 DeviceMinor:400 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a882ca4-1b11-494c-a51e-fb3c93ef5606/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:415 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes/kubernetes.io~projected/kube-api-access-f78rc DeviceMajor:0 DeviceMinor:271 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:277 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:277 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-429 DeviceMajor:0 DeviceMinor:429 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-435 DeviceMajor:0 DeviceMinor:435 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-441 DeviceMajor:0 DeviceMinor:441 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d9093009-6c67-4d6b-8d3a-a388dbb078a3/volumes/kubernetes.io~projected/kube-api-access-ldx4p DeviceMajor:0 DeviceMinor:64 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2fb7c085-6b91-409e-b15b-6d5b94f1c94c/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:231 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-446 DeviceMajor:0 DeviceMinor:446 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2fb7c085-6b91-409e-b15b-6d5b94f1c94c/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:229 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8d23da89396e0eb3763a357b85fe9e4bb4cdae904ab8265915f2ecb014d79cd0/userdata/shm DeviceMajor:0 DeviceMinor:237 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-395 DeviceMajor:0 DeviceMinor:395 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-158 DeviceMajor:0 DeviceMinor:158 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bb6f24bb-ba37-400e-878f-267797c1903b/volumes/kubernetes.io~projected/kube-api-access-6xtbl DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c134e4cc8deb953654e6b44f4087954e8815bd9e8a6aed22738757ff72a2f018/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-433 DeviceMajor:0 DeviceMinor:433 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/76ab3954-a12f-484a-b32f-3de7517bc70f/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:225 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/2fb7c085-6b91-409e-b15b-6d5b94f1c94c/volumes/kubernetes.io~projected/kube-api-access-xw7st DeviceMajor:0 DeviceMinor:234 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:368 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:363 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volume-subpaths/web-config/prometheus/15 DeviceMajor:0 DeviceMinor:368 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2d0e3b1f8b8dda90a7ff6b39e26ad3dcbbe6661ddbd61e1642bde37b87313fe6/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-141 DeviceMajor:0 DeviceMinor:141 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:280 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:282 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-405 DeviceMajor:0 DeviceMinor:405 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/592ebcb1e93b72d18af501402a503e32f6ec8bbf5c1019f4c15ba7b60040355b/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:373 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:419 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:2198be6303a3f26 MacAddress:e6:95:e4:68:b1:c6 Speed:10000 Mtu:8901} {Name:2279054b88b2fe1 MacAddress:f6:3b:48:95:75:b2 Speed:10000 Mtu:8901} {Name:3a3dec3c50c3664 MacAddress:d2:12:ce:7a:72:a8 Speed:10000 Mtu:8901} {Name:4f7a9b4759fb18f MacAddress:06:46:e8:af:6a:c3 Speed:10000 Mtu:8901} {Name:7539f30ff407b86 MacAddress:d2:9e:ce:28:9f:01 Speed:10000 Mtu:8901} {Name:7821eabf282976a MacAddress:02:c6:e1:0a:e4:c6 Speed:10000 Mtu:8901} {Name:8d23da89396e0eb MacAddress:b6:fa:b5:31:c6:66 Speed:10000 Mtu:8901} {Name:99392956d77083b MacAddress:ee:6b:95:5e:81:55 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c3:18:3e:ab Speed:0 Mtu:9001} {Name:br-int MacAddress:46:1a:ac:89:f6:79 Speed:0 Mtu:8901} {Name:e23ef180860396c MacAddress:7e:72:a5:58:e4:8c Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c3:18:3e:ab Speed:0 Mtu:9001} {Name:fba94a278e13d0b MacAddress:22:58:c7:a6:01:48 Speed:10000 Mtu:8901} {Name:fcdb921f5a03967 MacAddress:2e:8c:10:95:db:d2 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:52:27:b4:f3:97:fd Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:2a:62:48:84:9e:8d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.815887    5664 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.815955    5664 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.816304    5664 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.816335    5664 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-192.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.816537    5664 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.816548    5664 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.816566    5664 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.816593    5664 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 22:19:29.816919 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.816733    5664 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 22:19:29.817604 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.816974    5664 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 22:19:29.817604 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.817125    5664 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 22:19:29.817604 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.817170    5664 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 22:19:29.817604 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.817195    5664 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 22:19:29.817604 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.817210    5664 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 22:19:29.817604 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.817225    5664 kubelet.go:405] "Adding apiserver pod source"
Oct 30 22:19:29.817604 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.817237    5664 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 22:19:29.818087 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.818057    5664 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 22:19:29.818087 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.818072    5664 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:19:29.818629 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.818611    5664 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 22:19:29.821190 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821173    5664 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 22:19:29.821190 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821188    5664 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 22:19:29.821339 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821311    5664 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 22:19:29.821550 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821534    5664 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 22:19:29.821611 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821575    5664 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 22:19:29.821696 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821684    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 22:19:29.821750 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821705    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 22:19:29.821750 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821715    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 22:19:29.821750 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821724    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 22:19:29.821750 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821734    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 22:19:29.821750 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821744    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 22:19:29.822059 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821753    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 22:19:29.822059 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821763    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 22:19:29.822059 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821774    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 22:19:29.822059 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821793    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 22:19:29.822059 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821810    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 22:19:29.822059 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821826    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 22:19:29.822059 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821861    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 22:19:29.822059 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.821874    5664 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 22:19:29.825414 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.825398    5664 server.go:1263] "Started kubelet"
Oct 30 22:19:29.825923 ip-10-0-132-192 systemd[1]: Started Kubernetes Kubelet.
Oct 30 22:19:29.827938 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.825478    5664 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 22:19:29.827938 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.826185    5664 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 22:19:29.827938 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.826254    5664 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 22:19:29.828339 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.828250    5664 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 22:19:29.828669 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.828614    5664 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 22:19:29.834552 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.832773    5664 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 22:19:29.834552 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.832793    5664 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 22:19:29.834552 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.832864    5664 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 22:19:29.834552 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.832847    5664 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 22:13:20 +0000 UTC" deadline="2028-06-18 00:02:37.504807268 +0000 UTC"
Oct 30 22:19:29.834552 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.832874    5664 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 22:19:29.834552 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.832879    5664 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14305h43m7.671931689s"
Oct 30 22:19:29.834552 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:29.832938    5664 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-192.us-east-2.compute.internal\" not found"
Oct 30 22:19:29.834552 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.832968    5664 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 22:19:29.835446 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835419    5664 factory.go:55] Registering systemd factory
Oct 30 22:19:29.835446 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835447    5664 factory.go:223] Registration of the systemd container factory successfully
Oct 30 22:19:29.835616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835503    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr" seLinuxMountContext=""
Oct 30 22:19:29.835616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835540    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fb7c085-6b91-409e-b15b-6d5b94f1c94c" volumeName="kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 22:19:29.835616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835555    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41e14309-abd7-407f-b98b-39005ec7d563" volumeName="kubernetes.io/empty-dir/41e14309-abd7-407f-b98b-39005ec7d563-tmp-dir" seLinuxMountContext=""
Oct 30 22:19:29.835616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835568    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76ab3954-a12f-484a-b32f-3de7517bc70f" volumeName="kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-env-overrides" seLinuxMountContext=""
Oct 30 22:19:29.835616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835581    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b8cf69d-9ab1-4bfa-b927-509c0e847d1a" volumeName="kubernetes.io/empty-dir/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-volume-directive-shadow" seLinuxMountContext=""
Oct 30 22:19:29.835616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835593    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654" volumeName="kubernetes.io/projected/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-kube-api-access-h824h" seLinuxMountContext=""
Oct 30 22:19:29.835616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835604    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config" seLinuxMountContext=""
Oct 30 22:19:29.835616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835617    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b8cf69d-9ab1-4bfa-b927-509c0e847d1a" volumeName="kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835632    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835644    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835656    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6" volumeName="kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cni-binary-copy" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835668    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="41e14309-abd7-407f-b98b-39005ec7d563" volumeName="kubernetes.io/projected/41e14309-abd7-407f-b98b-39005ec7d563-kube-api-access-lqmvd" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835679    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72a3a151-21d2-4551-94b5-332f0b4f9542" volumeName="kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-binary-copy" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835693    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="777ef147-4f3f-4c89-ae2b-545aad6b31f1" volumeName="kubernetes.io/configmap/777ef147-4f3f-4c89-ae2b-545aad6b31f1-serviceca" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835709    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6695e08-7390-47f0-b5db-f474a466cf0a" volumeName="kubernetes.io/configmap/f6695e08-7390-47f0-b5db-f474a466cf0a-konnectivity-ca" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835732    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60e9435e-1582-493b-a73d-226b32c86abc" volumeName="kubernetes.io/projected/60e9435e-1582-493b-a73d-226b32c86abc-kube-api-access-4m8hc" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835744    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77422c7f-1d2c-4ca8-968c-3e59beffbb1e" volumeName="kubernetes.io/empty-dir/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-textfile" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835755    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f9eaa777-0ec3-4c87-92a6-05e46f50066f" volumeName="kubernetes.io/configmap/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835766    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835777    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76ab3954-a12f-484a-b32f-3de7517bc70f" volumeName="kubernetes.io/secret/76ab3954-a12f-484a-b32f-3de7517bc70f-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835791    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa" volumeName="kubernetes.io/projected/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-kube-api-access-lf9fr" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835801    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" volumeName="kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835813    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835825    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6" volumeName="kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-daemon-config" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835837    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7110fffb-3090-497a-a6d8-0260f3166ed1" volumeName="kubernetes.io/secret/7110fffb-3090-497a-a6d8-0260f3166ed1-metrics-tls" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835848    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77422c7f-1d2c-4ca8-968c-3e59beffbb1e" volumeName="kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835861    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654" volumeName="kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-tuned" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835875    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835912    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835925    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a882ca4-1b11-494c-a51e-fb3c93ef5606" volumeName="kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835937    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72a3a151-21d2-4551-94b5-332f0b4f9542" volumeName="kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835967    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa" volumeName="kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835981    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb6f24bb-ba37-400e-878f-267797c1903b" volumeName="kubernetes.io/configmap/bb6f24bb-ba37-400e-878f-267797c1903b-iptables-alerter-script" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.835993    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" volumeName="kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836005    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1" seLinuxMountContext=""
Oct 30 22:19:29.836032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836041    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836055    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b8cf69d-9ab1-4bfa-b927-509c0e847d1a" volumeName="kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836068    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b8cf69d-9ab1-4bfa-b927-509c0e847d1a" volumeName="kubernetes.io/projected/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-api-access-gng5d" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836081    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" volumeName="kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836093    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72a3a151-21d2-4551-94b5-332f0b4f9542" volumeName="kubernetes.io/projected/72a3a151-21d2-4551-94b5-332f0b4f9542-kube-api-access-cjdcx" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836106    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77422c7f-1d2c-4ca8-968c-3e59beffbb1e" volumeName="kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836118    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b8cf69d-9ab1-4bfa-b927-509c0e847d1a" volumeName="kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836121    5664 factory.go:153] Registering CRI-O factory
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836130    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" volumeName="kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836137    5664 factory.go:223] Registration of the crio container factory successfully
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836142    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654" volumeName="kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-tmp" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836157    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836169    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836180    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f6695e08-7390-47f0-b5db-f474a466cf0a" volumeName="kubernetes.io/secret/f6695e08-7390-47f0-b5db-f474a466cf0a-agent-certs" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836192    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836204    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04" volumeName="kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836218    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836232    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836244    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836256    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836268    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fb7c085-6b91-409e-b15b-6d5b94f1c94c" volumeName="kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836282    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76ab3954-a12f-484a-b32f-3de7517bc70f" volumeName="kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-config" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836296    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="777ef147-4f3f-4c89-ae2b-545aad6b31f1" volumeName="kubernetes.io/projected/777ef147-4f3f-4c89-ae2b-545aad6b31f1-kube-api-access-dlz2q" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836321    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" volumeName="kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836334    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836346    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836358    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fb7c085-6b91-409e-b15b-6d5b94f1c94c" volumeName="kubernetes.io/configmap/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836370    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76ab3954-a12f-484a-b32f-3de7517bc70f" volumeName="kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836194    5664 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836412    5664 factory.go:103] Registering Raw factory
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836430    5664 manager.go:1199] Started watching for new ooms in manager
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836738    5664 manager.go:319] Starting recovery of all containers
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836383    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77422c7f-1d2c-4ca8-968c-3e59beffbb1e" volumeName="kubernetes.io/projected/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-kube-api-access-mvw67" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836791    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" volumeName="kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836809    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836819    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836828    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836839    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836848    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836856    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6" volumeName="kubernetes.io/projected/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-kube-api-access-6jbvf" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836864    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="72a3a151-21d2-4551-94b5-332f0b4f9542" volumeName="kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836873    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" volumeName="kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836881    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836889    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f9eaa777-0ec3-4c87-92a6-05e46f50066f" volumeName="kubernetes.io/empty-dir/f9eaa777-0ec3-4c87-92a6-05e46f50066f-data-volume" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836927    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836939    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="60e9435e-1582-493b-a73d-226b32c86abc" volumeName="kubernetes.io/secret/60e9435e-1582-493b-a73d-226b32c86abc-cert" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836951    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7110fffb-3090-497a-a6d8-0260f3166ed1" volumeName="kubernetes.io/configmap/7110fffb-3090-497a-a6d8-0260f3166ed1-config-volume" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836964    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="76ab3954-a12f-484a-b32f-3de7517bc70f" volumeName="kubernetes.io/projected/76ab3954-a12f-484a-b32f-3de7517bc70f-kube-api-access-l9prx" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836976    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77422c7f-1d2c-4ca8-968c-3e59beffbb1e" volumeName="kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.836989    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d9093009-6c67-4d6b-8d3a-a388dbb078a3" volumeName="kubernetes.io/projected/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kube-api-access-ldx4p" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837000    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837013    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f9eaa777-0ec3-4c87-92a6-05e46f50066f" volumeName="kubernetes.io/secret/f9eaa777-0ec3-4c87-92a6-05e46f50066f-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837025    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837036    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837047    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="08103390-6605-487e-b4bb-9e367e7dda1f" volumeName="kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837059    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls" seLinuxMountContext=""
Oct 30 22:19:29.837027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837071    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2fb7c085-6b91-409e-b15b-6d5b94f1c94c" volumeName="kubernetes.io/projected/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-kube-api-access-xw7st" seLinuxMountContext=""
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837083    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7110fffb-3090-497a-a6d8-0260f3166ed1" volumeName="kubernetes.io/projected/7110fffb-3090-497a-a6d8-0260f3166ed1-kube-api-access-kb2jv" seLinuxMountContext=""
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837095    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77422c7f-1d2c-4ca8-968c-3e59beffbb1e" volumeName="kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-tls" seLinuxMountContext=""
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837115    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb6f24bb-ba37-400e-878f-267797c1903b" volumeName="kubernetes.io/projected/bb6f24bb-ba37-400e-878f-267797c1903b-kube-api-access-6xtbl" seLinuxMountContext=""
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837128    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg" seLinuxMountContext=""
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837139    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f9eaa777-0ec3-4c87-92a6-05e46f50066f" volumeName="kubernetes.io/projected/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-api-access-nspxc" seLinuxMountContext=""
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837151    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7110fffb-3090-497a-a6d8-0260f3166ed1" volumeName="kubernetes.io/empty-dir/7110fffb-3090-497a-a6d8-0260f3166ed1-tmp-dir" seLinuxMountContext=""
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837171    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9b8cf69d-9ab1-4bfa-b927-509c0e847d1a" volumeName="kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837184    5664 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" volumeName="kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837195    5664 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.837205    5664 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.839268    5664 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:19:29.840413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.839988    5664 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:19:29.843848 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.842930    5664 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 22:19:29.846569 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.846537    5664 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:19:29.860368 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.860335    5664 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 22:19:29.861690 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.861668    5664 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 22:19:29.861803 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.861697    5664 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 22:19:29.861803 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.861717    5664 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 22:19:29.861803 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:29.861755    5664 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 22:19:29.866396 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.866375    5664 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 22:19:29.879088 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.879061    5664 generic.go:356] "Generic (PLEG): container finished" podID="08103390-6605-487e-b4bb-9e367e7dda1f" containerID="8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e" exitCode=0
Oct 30 22:19:29.882824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.882804    5664 generic.go:356] "Generic (PLEG): container finished" podID="77422c7f-1d2c-4ca8-968c-3e59beffbb1e" containerID="cede5a467a81b66a674000268cdbb4fccae9ecf5b8918f3cbf077f7805524d39" exitCode=0
Oct 30 22:19:29.889986 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.889962    5664 generic.go:356] "Generic (PLEG): container finished" podID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc" exitCode=0
Oct 30 22:19:29.910588 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.910550    5664 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="d9a75a9e7aaabbf10d9551700d00f04223522c9474d54bfe9d1ab05c80a4639f" exitCode=0
Oct 30 22:19:29.910588 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.910585    5664 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="5328723b5757d58ab13c91a0ce6391477ca1bcf62c642a4a79ecdb5573581fc4" exitCode=0
Oct 30 22:19:29.910588 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.910594    5664 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="ef1357a6657bca3759f69e45fee96d9d887c0ef97ce2a78d5bc3f1c168545725" exitCode=0
Oct 30 22:19:29.910796 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.910601    5664 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="91fd29068dc3c84e90b8ab22c147cb4018a0c572683a3758fc381c68b83d589c" exitCode=0
Oct 30 22:19:29.910796 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.910608    5664 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="720ff1050746f0fe69f31839fd57240b95afa88d4f756f28f027e3cee5c313c8" exitCode=0
Oct 30 22:19:29.910796 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.910615    5664 generic.go:356] "Generic (PLEG): container finished" podID="72a3a151-21d2-4551-94b5-332f0b4f9542" containerID="a2727832b69128ea52d11ae8a387dc30a8eb158558bf0ae43d3e3715337dd6e9" exitCode=0
Oct 30 22:19:29.914124 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.914090    5664 generic.go:356] "Generic (PLEG): container finished" podID="222b3fa31d5bc6e9197fa2d0d2743eec" containerID="c9c80e7efc26debb770d345768e655064766e7edd41fe8dd533e67fe52af05c7" exitCode=0
Oct 30 22:19:29.945833 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.945768    5664 manager.go:324] Recovery completed
Oct 30 22:19:29.962425 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:29.962381    5664 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 22:19:29.970144 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970120    5664 cpu_manager.go:226] "Starting" policy="none"
Oct 30 22:19:29.970144 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970139    5664 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 22:19:29.970300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970156    5664 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 22:19:29.970300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970294    5664 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 22:19:29.970354 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970307    5664 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 22:19:29.970354 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970325    5664 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 22:19:29.970354 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970331    5664 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 22:19:29.970354 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970336    5664 policy_none.go:50] "Start"
Oct 30 22:19:29.970354 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970345    5664 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 22:19:29.970354 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970355    5664 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 22:19:29.970518 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970486    5664 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 22:19:29.970518 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970493    5664 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 22:19:29.970518 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.970498    5664 policy_none.go:44] "Start"
Oct 30 22:19:29.975152 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.975115    5664 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 22:19:29.977691 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.977678    5664 manager.go:341] "Starting Device Plugin manager"
Oct 30 22:19:29.978293 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:29.977704    5664 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 22:19:29.978347 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.978297    5664 server.go:87] "Starting device plugin registration server"
Oct 30 22:19:29.978563 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.978552    5664 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 22:19:29.978594 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.978564    5664 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 22:19:29.978686 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.978662    5664 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 22:19:29.978820 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.978768    5664 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 22:19:29.978820 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.978811    5664 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 22:19:29.978820 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.978817    5664 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 22:19:29.979064 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.978869    5664 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T22:19:29.978798951Z","UUID":"5fe6ea9b-c3b2-4f30-beed-6bfcac28089e","Handler":null,"Name":"","Endpoint":""}
Oct 30 22:19:29.980393 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.980372    5664 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 22:19:29.980497 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:29.980401    5664 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 22:19:29.980554 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:29.980516    5664 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 22:19:30.078997 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.078961    5664 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 22:19:30.081124 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.081103    5664 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 22:19:30.081202 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.081137    5664 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 22:19:30.081202 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.081152    5664 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-192.us-east-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 22:19:30.081202 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.081197    5664 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.094838 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.094818    5664 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.094948 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.094920    5664 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.162920 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.162837    5664 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"]
Oct 30 22:19:30.163608 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.163586    5664 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.163723 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.163627    5664 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.163769 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.163690    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"320724d28f4bb4eb7790f5cce736ad83","Type":"ContainerStarted","Data":"74f57e514d45c7b61590d6523f1c8059dfa300b42afcc5901c81f151ef90cf5e"}
Oct 30 22:19:30.163769 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.163749    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"320724d28f4bb4eb7790f5cce736ad83","Type":"ContainerStarted","Data":"13fbf1123d6b450a07f082ff67b0543ec582413e69ce9216baaaa4c443160063"}
Oct 30 22:19:30.163867 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.163806    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"222b3fa31d5bc6e9197fa2d0d2743eec","Type":"ContainerStarted","Data":"0b45b9b038a49461f0288eaf4bbbeea28886fc94ab02ca1f7ce125b35ed9d57a"}
Oct 30 22:19:30.163867 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.163820    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"222b3fa31d5bc6e9197fa2d0d2743eec","Type":"ContainerDied","Data":"c9c80e7efc26debb770d345768e655064766e7edd41fe8dd533e67fe52af05c7"}
Oct 30 22:19:30.163867 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.163832    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal" event={"ID":"222b3fa31d5bc6e9197fa2d0d2743eec","Type":"ContainerStarted","Data":"ddd09928a7a0cd62a016a44fcc552e3de824cf3a711ed70bddf0e9b82e669f35"}
Oct 30 22:19:30.176869 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.176837    5664 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:19:30.176972 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.176919    5664 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.177832 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.177806    5664 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:19:30.177944 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.177845    5664 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.245247 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.245147    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.245247 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.245197    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/320724d28f4bb4eb7790f5cce736ad83-config\") pod \"kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"320724d28f4bb4eb7790f5cce736ad83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.245247 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.245216    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.345830 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.345797    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/320724d28f4bb4eb7790f5cce736ad83-config\") pod \"kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"320724d28f4bb4eb7790f5cce736ad83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.345959 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.345835    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.345959 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.345862    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.345959 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.345923    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/320724d28f4bb4eb7790f5cce736ad83-config\") pod \"kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"320724d28f4bb4eb7790f5cce736ad83\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.345959 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.345941    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.346077 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.345926    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/222b3fa31d5bc6e9197fa2d0d2743eec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" (UID: \"222b3fa31d5bc6e9197fa2d0d2743eec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.817631 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.817574    5664 apiserver.go:52] "Watching apiserver"
Oct 30 22:19:30.829009 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.828983    5664 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 22:19:30.829763 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.829739    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-gbckc","openshift-cluster-node-tuning-operator/tuned-ht6mr","openshift-multus/multus-4bwqs","openshift-monitoring/prometheus-k8s-0","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk","openshift-console/console-688b666f4d-5pmrq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal","openshift-multus/network-metrics-daemon-cdmr4","openshift-ovn-kubernetes/ovnkube-node-hdm75","openshift-insights/insights-runtime-extractor-6w8dk","kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal","openshift-ingress-canary/ingress-canary-gt7k2","openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz","openshift-monitoring/node-exporter-2j5mw","openshift-network-operator/iptables-alerter-h6x2s","openshift-monitoring/alertmanager-main-0","openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8","kube-system/global-pull-secret-syncer-bzxzm","openshift-network-diagnostics/network-check-target-27nh7","openshift-dns/dns-default-wghlv","openshift-image-registry/node-ca-kx7l5","kube-system/konnectivity-agent-ptpfs","openshift-dns/node-resolver-cf4tj"]
Oct 30 22:19:30.837324 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837298    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 22:19:30.837505 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837490    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-sdfwl\""
Oct 30 22:19:30.837606 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837520    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jb8sb\""
Oct 30 22:19:30.837606 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837553    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-szxgd\""
Oct 30 22:19:30.837701 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837663    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-4x7f7\""
Oct 30 22:19:30.838015 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837818    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kkwnr\""
Oct 30 22:19:30.838015 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837849    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 22:19:30.838015 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837855    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jm6gb\""
Oct 30 22:19:30.838015 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837854    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms5nl\""
Oct 30 22:19:30.838015 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837920    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-glgqx\""
Oct 30 22:19:30.838015 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.837977    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 22:19:30.838345 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.838059    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-r4dw4\""
Oct 30 22:19:30.839136 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.838831    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 22:19:30.839505 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.839484    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 22:19:30.840128 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.840098    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.840834 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.840810    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 22:19:30.841385 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.841361    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 22:19:30.841547 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.841520    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.841798 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.841781    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.842735 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.842711    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.842845 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.842830    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 22:19:30.843018 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.842996    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 22:19:30.843123 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.843107    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.843190 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.843174    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 22:19:30.843391 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.843369    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 22:19:30.843448 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.843183    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.844143 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844104    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 22:19:30.844246 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844225    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 22:19:30.844310 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844286    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.844356 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844324    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.844453 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844435    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.844604 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844581    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.844719 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844606    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.844719 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844644    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 22:19:30.844858 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844841    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.844982 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844954    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 22:19:30.845042 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.845018    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 22:19:30.845333 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.844226    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.845434 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.845351    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 22:19:30.845434 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.845382    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-tp94j\""
Oct 30 22:19:30.845840 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.845542    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.845840 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.845598    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 22:19:30.846586 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.846563    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.846701 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.846630    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.846701 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.846681    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 22:19:30.846805 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.846726    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.846805 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.846743    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 22:19:30.846805 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.846794    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.847006 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.846828    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-qwcf4\""
Oct 30 22:19:30.847161 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847141    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 22:19:30.847258 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847179    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 22:19:30.847258 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847212    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 22:19:30.847353 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847278    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 22:19:30.847353 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847284    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-cnibin\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.847353 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847303    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-jfcbv\""
Oct 30 22:19:30.847353 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847321    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41e14309-abd7-407f-b98b-39005ec7d563-tmp-dir\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:19:30.847536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847356    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.847536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847368    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 22:19:30.847536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847354    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:19:30.847536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847412    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 22:19:30.847536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847431    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 22:19:30.847536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847457    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 22:19:30.847536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847412    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-bin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.847536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847512    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlz2q\" (UniqueName: \"kubernetes.io/projected/777ef147-4f3f-4c89-ae2b-545aad6b31f1-kube-api-access-dlz2q\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:19:30.847536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847528    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847541    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-bin\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847571    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847570    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-etc-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847595    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-jmj7l\""
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847607    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-srblb\""
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847616    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/777ef147-4f3f-4c89-ae2b-545aad6b31f1-serviceca\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847621    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-metrics-certs\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847546    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/41e14309-abd7-407f-b98b-39005ec7d563-tmp-dir\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847641    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847675    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847732    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldx4p\" (UniqueName: \"kubernetes.io/projected/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kube-api-access-ldx4p\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847745    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bh4rp\""
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847764    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-multus\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847789    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-kubelet\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847814    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9prx\" (UniqueName: \"kubernetes.io/projected/76ab3954-a12f-484a-b32f-3de7517bc70f-kube-api-access-l9prx\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.847934 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847857    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/777ef147-4f3f-4c89-ae2b-545aad6b31f1-host\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.847978    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-sys-fs\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848034    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848062    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41e14309-abd7-407f-b98b-39005ec7d563-hosts-file\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848086    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-socket-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848133    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-registration-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848214    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-multus-certs\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848225    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848239    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-env-overrides\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848328    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-cg9xt\""
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848364    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9jqsq\""
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848376    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-device-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848406    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-m6d59\""
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848421    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848436    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/777ef147-4f3f-4c89-ae2b-545aad6b31f1-serviceca\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:19:30.848570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848445    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-env-overrides\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848608    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-kubelet-config\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848652    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-slash\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848649    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848709    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-netns\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848777    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-netns\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848844    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-systemd-units\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848887    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848941    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-system-cni-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.848980    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-binary-copy\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849003    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.849082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849025    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-netd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849386 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849099    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cnibin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.849386 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849123    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76ab3954-a12f-484a-b32f-3de7517bc70f-ovn-node-metrics-cert\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849386 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849148    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-script-lib\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849386 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849170    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.849386 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849194    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf9fr\" (UniqueName: \"kubernetes.io/projected/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-kube-api-access-lf9fr\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:19:30.849386 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849278    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:30.849386 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849344    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-var-lib-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849659 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849633    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-ovn\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849793 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849684    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-node-log\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.849793 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849723    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-conf-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.849923 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849785    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-etc-kubernetes\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.849923 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849803    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1a882ca4-1b11-494c-a51e-fb3c93ef5606-original-pull-secret\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:30.849923 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849827    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.849923 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849882    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 22:19:30.849923 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849882    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjdcx\" (UniqueName: \"kubernetes.io/projected/72a3a151-21d2-4551-94b5-332f0b4f9542-kube-api-access-cjdcx\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.850176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849952    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/76ab3954-a12f-484a-b32f-3de7517bc70f-ovn-node-metrics-cert\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.850176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.849953    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6695e08-7390-47f0-b5db-f474a466cf0a-konnectivity-ca\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:19:30.850176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850091    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/72a3a151-21d2-4551-94b5-332f0b4f9542-cni-binary-copy\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.850176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850094    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-log-socket\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.850176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850151    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-script-lib\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.850176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850154    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:19:30.850464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850197    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-dbus\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:30.850464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850197    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f6695e08-7390-47f0-b5db-f474a466cf0a-konnectivity-ca\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:19:30.850464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850221    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6695e08-7390-47f0-b5db-f474a466cf0a-agent-certs\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:19:30.850464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850262    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cni-binary-copy\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.850464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850308    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-socket-dir-parent\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.850464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850405    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-daemon-config\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.850464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850422    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-systemd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.850464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850430    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f6695e08-7390-47f0-b5db-f474a466cf0a-agent-certs\") pod \"konnectivity-agent-ptpfs\" (UID: \"f6695e08-7390-47f0-b5db-f474a466cf0a\") " pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:19:30.850464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850460    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cni-binary-copy\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850471    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850504    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-config\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850521    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-system-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850595    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-k8s-cni-cncf-io\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850604    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-daemon-config\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850637    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-kubelet\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850685    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850690    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/76ab3954-a12f-484a-b32f-3de7517bc70f-ovnkube-config\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850728    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqmvd\" (UniqueName: \"kubernetes.io/projected/41e14309-abd7-407f-b98b-39005ec7d563-kube-api-access-lqmvd\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850752    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-hostroot\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.850936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.850769    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jbvf\" (UniqueName: \"kubernetes.io/projected/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-kube-api-access-6jbvf\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.860420 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.860401    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlz2q\" (UniqueName: \"kubernetes.io/projected/777ef147-4f3f-4c89-ae2b-545aad6b31f1-kube-api-access-dlz2q\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:19:30.862176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.862155    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjdcx\" (UniqueName: \"kubernetes.io/projected/72a3a151-21d2-4551-94b5-332f0b4f9542-kube-api-access-cjdcx\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.863290 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.863274    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9prx\" (UniqueName: \"kubernetes.io/projected/76ab3954-a12f-484a-b32f-3de7517bc70f-kube-api-access-l9prx\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.864345 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.864329    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldx4p\" (UniqueName: \"kubernetes.io/projected/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kube-api-access-ldx4p\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.865618 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.865602    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrdzx\" (UniqueName: \"kubernetes.io/projected/3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04-kube-api-access-xrdzx\") pod \"network-check-target-27nh7\" (UID: \"3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04\") " pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:19:30.867315 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.867299    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqmvd\" (UniqueName: \"kubernetes.io/projected/41e14309-abd7-407f-b98b-39005ec7d563-kube-api-access-lqmvd\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:19:30.868979 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.868959    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf9fr\" (UniqueName: \"kubernetes.io/projected/7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa-kube-api-access-lf9fr\") pod \"network-metrics-daemon-cdmr4\" (UID: \"7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa\") " pod="openshift-multus/network-metrics-daemon-cdmr4"
Oct 30 22:19:30.869948 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.869931    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jbvf\" (UniqueName: \"kubernetes.io/projected/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-kube-api-access-6jbvf\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.925803 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.925775    5664 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.925956 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.925920    5664 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.934831 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.934815    5664 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 22:19:30.940980 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.940956    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 22:19:30.940980 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.940969    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 22:19:30.941177 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941159    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 22:19:30.941251 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941228    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 22:19:30.941393 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941371    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 22:19:30.941460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941415    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 22:19:30.941460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941428    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 22:19:30.941460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941422    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-x6d62\""
Oct 30 22:19:30.941460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941459    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 22:19:30.941639 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941492    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.941858 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941837    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 22:19:30.941858 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941838    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 22:19:30.942033 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941867    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 22:19:30.942033 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941887    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 22:19:30.942033 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941886    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 22:19:30.942033 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.941943    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 22:19:30.942291 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942230    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 22:19:30.942291 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942269    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-caus6766sf7av\""
Oct 30 22:19:30.942291 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942275    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 22:19:30.942423 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942306    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 22:19:30.942423 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942309    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 22:19:30.942423 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942293    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 22:19:30.942423 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942313    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 22:19:30.942423 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942409    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4f76b\""
Oct 30 22:19:30.942661 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942477    5664 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:19:30.942661 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.942518    5664 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.942748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.942671    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 22:19:30.945554 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.945531    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 22:19:30.946917 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.946872    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 22:19:30.947073 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.947058    5664 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 22:19:30.947151 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.947118    5664 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 22:19:30.951041 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951008    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/777ef147-4f3f-4c89-ae2b-545aad6b31f1-host\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:19:30.951041 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951035    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-sys-fs\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.951204 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951053    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.951204 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951069    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.951204 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951086    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.951204 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951100    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7110fffb-3090-497a-a6d8-0260f3166ed1-metrics-tls\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:30.951204 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951145    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-sys-fs\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.951204 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951171    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/777ef147-4f3f-4c89-ae2b-545aad6b31f1-host\") pod \"node-ca-kx7l5\" (UID: \"777ef147-4f3f-4c89-ae2b-545aad6b31f1\") " pod="openshift-image-registry/node-ca-kx7l5"
Oct 30 22:19:30.951204 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951201    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951230    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-multus-certs\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951203    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951255    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951260    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951282    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-sys\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951304    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-multus-certs\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.951344    5664 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951306    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-kubelet-config\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951370    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951375    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-kubelet-config\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951392    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7110fffb-3090-497a-a6d8-0260f3166ed1-metrics-tls\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951412    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-slash\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.951478    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:31.451446118 +0000 UTC m=+1.702909587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951497    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951499    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-slash\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.951526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951530    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-netns\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951629    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951646    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-netns\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951678    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vgr\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951676    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951710    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951738    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-system-cni-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951754    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951792    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-netd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951814    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951813    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-system-cni-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951844    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-netd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951857    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-textfile\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951882    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvw67\" (UniqueName: \"kubernetes.io/projected/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-kube-api-access-mvw67\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951926    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951951    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951965    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-textfile\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.951972    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952004    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952030    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-var-lib-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952054    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952080    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952105    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952116    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-var-lib-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952127    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-metrics-client-ca\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.952153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952151    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-etc-kubernetes\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952174    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952182    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952195    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-log-socket\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952204    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952207    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952226    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9eaa777-0ec3-4c87-92a6-05e46f50066f-data-volume\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952233    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-log-socket\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952257    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952263    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952275    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-etc-kubernetes\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952282    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9eaa777-0ec3-4c87-92a6-05e46f50066f-data-volume\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952295    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952312    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952321    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-wtmp\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952331    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952350    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952376    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-metrics-client-ca\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.952380    5664 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952386    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-dbus\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952420    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952449    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1a882ca4-1b11-494c-a51e-fb3c93ef5606-dbus\") pod \"global-pull-secret-syncer-bzxzm\" (UID: \"1a882ca4-1b11-494c-a51e-fb3c93ef5606\") " pod="kube-system/global-pull-secret-syncer-bzxzm"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.952453    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:31.452437435 +0000 UTC m=+1.703900903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"service-ca" not registered
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952489    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952492    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952521    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952527    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952528    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952545    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-modprobe-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952544    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952553    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952580    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-lib-modules\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952608    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952641    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gng5d\" (UniqueName: \"kubernetes.io/projected/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-api-access-gng5d\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952661    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnlcg\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952678    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysconfig\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952698    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-k8s-cni-cncf-io\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952735    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-run-k8s-cni-cncf-io\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952774    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952795    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-kubelet\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952840    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952869    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-kubelet\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952867    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-var-lib-kubelet\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952928    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:30.952946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952935    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-ovn-kubernetes\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.952981    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953010    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953035    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-hostroot\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953062    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-conf\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953089    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953093    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-hostroot\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953113    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953117    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953115    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eaa777-0ec3-4c87-92a6-05e46f50066f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953158    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953183    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-bin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953186    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953208    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-bin\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953235    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-run\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953239    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-bin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.953249    5664 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953249    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953257    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-host\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953277    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-cni-bin\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953283    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.953308    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:31.453297411 +0000 UTC m=+1.704760883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953309    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9eaa777-0ec3-4c87-92a6-05e46f50066f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953360    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953404    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nspxc\" (UniqueName: \"kubernetes.io/projected/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-api-access-nspxc\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953436    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953438    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953434    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953539    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953558    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953582    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41e14309-abd7-407f-b98b-39005ec7d563-hosts-file\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953594    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953614    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953636    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953658    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953682    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/41e14309-abd7-407f-b98b-39005ec7d563-hosts-file\") pod \"node-resolver-cf4tj\" (UID: \"41e14309-abd7-407f-b98b-39005ec7d563\") " pod="openshift-dns/node-resolver-cf4tj"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953686    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-socket-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953725    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-registration-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953741    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-socket-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953751    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953772    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953778    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-device-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953805    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-registration-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953826    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953811    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-netns\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953842    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953845    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-host-run-netns\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953869    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-tmp\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953884    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-device-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953912    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m8hc\" (UniqueName: \"kubernetes.io/projected/60e9435e-1582-493b-a73d-226b32c86abc-kube-api-access-4m8hc\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953872    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953941    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-root\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953946    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-tmp\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.953976    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954031    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954086    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-systemd-units\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954117    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-systemd\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954136    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-tls\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954151    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-systemd-units\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954158    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954186    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954190    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954206    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954222    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954363    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cnibin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954380    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-tls\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954387    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb6f24bb-ba37-400e-878f-267797c1903b-iptables-alerter-script\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954440    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-cnibin\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954452    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954472    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954503    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954519    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bb6f24bb-ba37-400e-878f-267797c1903b-iptables-alerter-script\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954527    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954555    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-ovn\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.954573    5664 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954578    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d9093009-6c67-4d6b-8d3a-a388dbb078a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-849dk\" (UID: \"d9093009-6c67-4d6b-8d3a-a388dbb078a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-849dk"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954579    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-node-log\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954618    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb6f24bb-ba37-400e-878f-267797c1903b-host-slash\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.954630    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:31.454613031 +0000 UTC m=+1.706076478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-config" not registered
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954644    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-node-log\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954662    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw7st\" (UniqueName: \"kubernetes.io/projected/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-kube-api-access-xw7st\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954686    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-ovn\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.954664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954693    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954740    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954842    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-conf-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954892    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-conf-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954954    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60e9435e-1582-493b-a73d-226b32c86abc-cert\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954980    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.954983    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955011    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955028    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-socket-dir-parent\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955044    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-systemd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955059    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-sys\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955075    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955089    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955117    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9eaa777-0ec3-4c87-92a6-05e46f50066f-crio-socket\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955132    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7110fffb-3090-497a-a6d8-0260f3166ed1-tmp-dir\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955147    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb2jv\" (UniqueName: \"kubernetes.io/projected/7110fffb-3090-497a-a6d8-0260f3166ed1-kube-api-access-kb2jv\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955166    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955179    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955192    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955208    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-system-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955225    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xtbl\" (UniqueName: \"kubernetes.io/projected/bb6f24bb-ba37-400e-878f-267797c1903b-kube-api-access-6xtbl\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955240    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-kubernetes\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955255    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-tuned\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955269    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h824h\" (UniqueName: \"kubernetes.io/projected/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-kube-api-access-h824h\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955286    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955300    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955314    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955336    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-cnibin\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955353    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955368    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955385    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-etc-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955391    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/60e9435e-1582-493b-a73d-226b32c86abc-cert\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955400    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-multus\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955424    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-kubelet\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955432    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-cni-multus\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955452    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955466    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-system-cni-dir\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955480    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7110fffb-3090-497a-a6d8-0260f3166ed1-config-volume\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955535    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955590    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-multus-socket-dir-parent\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955631    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-run-systemd\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955671    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-tuned\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955682    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7110fffb-3090-497a-a6d8-0260f3166ed1-config-volume\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955853    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955873    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.955911    5664 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955367    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955961    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/76ab3954-a12f-484a-b32f-3de7517bc70f-etc-openvswitch\") pod \"ovnkube-node-hdm75\" (UID: \"76ab3954-a12f-484a-b32f-3de7517bc70f\") " pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955873    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.955971    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7110fffb-3090-497a-a6d8-0260f3166ed1-tmp-dir\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.956003    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6-host-var-lib-kubelet\") pod \"multus-4bwqs\" (UID: \"1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6\") " pod="openshift-multus/multus-4bwqs"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.956104    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:31.456079588 +0000 UTC m=+1.707543035 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.956114    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/72a3a151-21d2-4551-94b5-332f0b4f9542-cnibin\") pod \"multus-additional-cni-plugins-gbckc\" (UID: \"72a3a151-21d2-4551-94b5-332f0b4f9542\") " pod="openshift-multus/multus-additional-cni-plugins-gbckc"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.956112    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.956133    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.956199    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-accelerators-collector-config\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.956279    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.956286    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.957328 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.956319    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.968313 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.968289    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvw67\" (UniqueName: \"kubernetes.io/projected/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-kube-api-access-mvw67\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:30.968382 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.968337    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nspxc\" (UniqueName: \"kubernetes.io/projected/f9eaa777-0ec3-4c87-92a6-05e46f50066f-kube-api-access-nspxc\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:30.968424 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.968391    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gng5d\" (UniqueName: \"kubernetes.io/projected/9b8cf69d-9ab1-4bfa-b927-509c0e847d1a-kube-api-access-gng5d\") pod \"kube-state-metrics-5dc8d7d89b-p4pkz\" (UID: \"9b8cf69d-9ab1-4bfa-b927-509c0e847d1a\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-p4pkz"
Oct 30 22:19:30.968474 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.968460    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vgr\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr\") pod \"alertmanager-main-0\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:30.969364 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.969349    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb2jv\" (UniqueName: \"kubernetes.io/projected/7110fffb-3090-497a-a6d8-0260f3166ed1-kube-api-access-kb2jv\") pod \"dns-default-wghlv\" (UID: \"7110fffb-3090-497a-a6d8-0260f3166ed1\") " pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:30.972093 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.972078    5664 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 22:19:30.972138 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.972095    5664 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 22:19:30.972138 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.972106    5664 projected.go:196] Error preparing data for projected volume kube-api-access-f78rc for pod openshift-console/console-688b666f4d-5pmrq: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:30.972191 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:30.972168    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:31.472155627 +0000 UTC m=+1.723619090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-f78rc" (UniqueName: "kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:30.973143 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.973128    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h824h\" (UniqueName: \"kubernetes.io/projected/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-kube-api-access-h824h\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:30.973461 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.973446    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xtbl\" (UniqueName: \"kubernetes.io/projected/bb6f24bb-ba37-400e-878f-267797c1903b-kube-api-access-6xtbl\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:19:30.973725 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.973707    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw7st\" (UniqueName: \"kubernetes.io/projected/2fb7c085-6b91-409e-b15b-6d5b94f1c94c-kube-api-access-xw7st\") pod \"openshift-state-metrics-7bd7489957-l6wf8\" (UID: \"2fb7c085-6b91-409e-b15b-6d5b94f1c94c\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-l6wf8"
Oct 30 22:19:30.974380 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.974364    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnlcg\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg\") pod \"prometheus-k8s-0\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:30.974501 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:30.974485    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m8hc\" (UniqueName: \"kubernetes.io/projected/60e9435e-1582-493b-a73d-226b32c86abc-kube-api-access-4m8hc\") pod \"ingress-canary-gt7k2\" (UID: \"60e9435e-1582-493b-a73d-226b32c86abc\") " pod="openshift-ingress-canary/ingress-canary-gt7k2"
Oct 30 22:19:31.056887 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.056854    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-sys\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057086 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.056889    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9eaa777-0ec3-4c87-92a6-05e46f50066f-crio-socket\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:31.057086 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.056938    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-kubernetes\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057086 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.056983    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057086 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057024    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-sys\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:31.057086 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057055    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-sys\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:31.057086 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057024    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-kubernetes\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057086 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057082    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-wtmp\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057055    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9eaa777-0ec3-4c87-92a6-05e46f50066f-crio-socket\") pod \"insights-runtime-extractor-6w8dk\" (UID: \"f9eaa777-0ec3-4c87-92a6-05e46f50066f\") " pod="openshift-insights/insights-runtime-extractor-6w8dk"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.056981    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-sys\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057109    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057143    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-modprobe-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057160    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-lib-modules\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057178    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysconfig\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057196    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-var-lib-kubelet\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057214    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-conf\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057179    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-node-exporter-wtmp\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057246    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-modprobe-d\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057256    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysconfig\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057278    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-run\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057283    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-var-lib-kubelet\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057312    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-host\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057317    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-lib-modules\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057313    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-run\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057334    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-sysctl-conf\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057342    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-host\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057371    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-root\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057388    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-systemd\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057418    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77422c7f-1d2c-4ca8-968c-3e59beffbb1e-root\") pod \"node-exporter-2j5mw\" (UID: \"77422c7f-1d2c-4ca8-968c-3e59beffbb1e\") " pod="openshift-monitoring/node-exporter-2j5mw"
Oct 30 22:19:31.057426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057424    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654-etc-systemd\") pod \"tuned-ht6mr\" (UID: \"dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654\") " pod="openshift-cluster-node-tuning-operator/tuned-ht6mr"
Oct 30 22:19:31.058528 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057452    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb6f24bb-ba37-400e-878f-267797c1903b-host-slash\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:19:31.058528 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.057505    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bb6f24bb-ba37-400e-878f-267797c1903b-host-slash\") pod \"iptables-alerter-h6x2s\" (UID: \"bb6f24bb-ba37-400e-878f-267797c1903b\") " pod="openshift-network-operator/iptables-alerter-h6x2s"
Oct 30 22:19:31.460518 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.460478    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:31.460518 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.460524    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:31.460726 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.460550    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:31.460726 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.460588    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:31.460726 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460613    5664 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 22:19:31.460726 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460630    5664 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:31.460726 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460676    5664 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 22:19:31.460726 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460693    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:32.460673612 +0000 UTC m=+2.712137063 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"service-ca" not registered
Oct 30 22:19:31.460726 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460699    5664 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:31.460726 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460716    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:32.460702728 +0000 UTC m=+2.712166174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-config" not registered
Oct 30 22:19:31.460983 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.460754    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:31.460983 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460795    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:32.460787975 +0000 UTC m=+2.712251420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:31.460983 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460809    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:32.460803486 +0000 UTC m=+2.712266932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:31.460983 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460833    5664 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:31.460983 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.460866    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:32.460856675 +0000 UTC m=+2.712320138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:31.561986 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:31.561954    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:31.562125 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.562112    5664 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 22:19:31.562158 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.562134    5664 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 22:19:31.562158 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.562146    5664 projected.go:196] Error preparing data for projected volume kube-api-access-f78rc for pod openshift-console/console-688b666f4d-5pmrq: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:31.562213 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:31.562196    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:32.562183451 +0000 UTC m=+2.813646911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78rc" (UniqueName: "kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:32.470798 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:32.470758    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:32.470810    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:32.470857    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:32.470884    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.470885    5664 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:32.470938    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.470974    5664 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.470992    5664 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.470999    5664 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.471032    5664 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.471040    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:34.471013319 +0000 UTC m=+4.722476765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-config" not registered
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.471068    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:34.471048065 +0000 UTC m=+4.722511533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.471087    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:34.471077127 +0000 UTC m=+4.722540576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"service-ca" not registered
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.471098    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:34.471093012 +0000 UTC m=+4.722556457 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:32.471220 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.471108    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:34.471103344 +0000 UTC m=+4.722566790 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:32.572050 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:32.572010    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:32.572228 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.572203    5664 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 22:19:32.572283 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.572228    5664 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 22:19:32.572283 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.572241    5664 projected.go:196] Error preparing data for projected volume kube-api-access-f78rc for pod openshift-console/console-688b666f4d-5pmrq: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:32.572402 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:32.572313    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:34.572294408 +0000 UTC m=+4.823757874 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78rc" (UniqueName: "kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:33.706442 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:33.706409    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:33.914948 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:33.914921    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wghlv"
Oct 30 22:19:34.493287 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:34.493238    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:34.493479 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:34.493301    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:34.493479 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:34.493348    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:34.493479 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493376    5664 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:34.493479 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:34.493389    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:34.493479 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:34.493432    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:34.493479 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493448    5664 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:34.493778 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493375    5664 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 22:19:34.493778 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493509    5664 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:34.493778 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493521    5664 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 22:19:34.493778 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493453    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:38.493438871 +0000 UTC m=+8.744902345 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:34.493778 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493597    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:38.493582483 +0000 UTC m=+8.745045930 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:34.493778 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493617    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:38.493607141 +0000 UTC m=+8.745070592 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-config" not registered
Oct 30 22:19:34.493778 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493643    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:38.493633252 +0000 UTC m=+8.745096701 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"service-ca" not registered
Oct 30 22:19:34.493778 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.493659    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:38.493650972 +0000 UTC m=+8.745114424 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:34.594504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:34.594465    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:34.594665 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.594621    5664 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 22:19:34.594665 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.594642    5664 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 22:19:34.594665 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.594651    5664 projected.go:196] Error preparing data for projected volume kube-api-access-f78rc for pod openshift-console/console-688b666f4d-5pmrq: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:34.594796 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:34.594706    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:38.594689937 +0000 UTC m=+8.846153383 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78rc" (UniqueName: "kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:36.428257 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:36.428225    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:36.487963 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:36.487936    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:36.950733 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:36.947997    5664 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 22:19:37.515666 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:37.515630    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:19:38.533813 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:38.533772    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:38.533835    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:38.533862    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.533943    5664 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:38.533964    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.533984    5664 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.533943    5664 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.534010    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:46.533993347 +0000 UTC m=+16.785456793 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:38.534035    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.534051    5664 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.534075    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:46.534056123 +0000 UTC m=+16.785519589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.534086    5664 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.534098    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:46.534088603 +0000 UTC m=+16.785552050 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.534113    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:46.534106457 +0000 UTC m=+16.785569903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-config" not registered
Oct 30 22:19:38.534332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.534124    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:46.534118366 +0000 UTC m=+16.785581815 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"service-ca" not registered
Oct 30 22:19:38.635531 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:38.635485    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:38.635719 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.635648    5664 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 22:19:38.635719 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.635667    5664 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 22:19:38.635719 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.635678    5664 projected.go:196] Error preparing data for projected volume kube-api-access-f78rc for pod openshift-console/console-688b666f4d-5pmrq: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:38.635823 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:38.635743    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:19:46.635727136 +0000 UTC m=+16.887190582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78rc" (UniqueName: "kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:39.255419 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:39.255386    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:19:42.569829 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.569795    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:19:42.570331 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.570182    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy-metric" containerID="cri-o://591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11" gracePeriod=120
Oct 30 22:19:42.570331 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.570258    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="config-reloader" containerID="cri-o://8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef" gracePeriod=120
Oct 30 22:19:42.570331 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.570297    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="prom-label-proxy" containerID="cri-o://af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e" gracePeriod=120
Oct 30 22:19:42.570498 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.570161    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="alertmanager" containerID="cri-o://fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012" gracePeriod=120
Oct 30 22:19:42.570498 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.570400    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy" containerID="cri-o://74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351" gracePeriod=120
Oct 30 22:19:42.570498 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.570427    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy-web" containerID="cri-o://cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476" gracePeriod=120
Oct 30 22:19:42.974761 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974730    5664 generic.go:356] "Generic (PLEG): container finished" podID="08103390-6605-487e-b4bb-9e367e7dda1f" containerID="af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e" exitCode=0
Oct 30 22:19:42.974761 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974753    5664 generic.go:356] "Generic (PLEG): container finished" podID="08103390-6605-487e-b4bb-9e367e7dda1f" containerID="591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11" exitCode=0
Oct 30 22:19:42.974761 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974760    5664 generic.go:356] "Generic (PLEG): container finished" podID="08103390-6605-487e-b4bb-9e367e7dda1f" containerID="74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351" exitCode=0
Oct 30 22:19:42.974761 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974765    5664 generic.go:356] "Generic (PLEG): container finished" podID="08103390-6605-487e-b4bb-9e367e7dda1f" containerID="8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef" exitCode=0
Oct 30 22:19:42.975090 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974771    5664 generic.go:356] "Generic (PLEG): container finished" podID="08103390-6605-487e-b4bb-9e367e7dda1f" containerID="fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012" exitCode=0
Oct 30 22:19:42.975090 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974810    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerDied","Data":"af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e"}
Oct 30 22:19:42.975090 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974891    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerDied","Data":"591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11"}
Oct 30 22:19:42.975090 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974943    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerDied","Data":"74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351"}
Oct 30 22:19:42.975090 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974967    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerDied","Data":"8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef"}
Oct 30 22:19:42.975090 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:42.974989    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerDied","Data":"fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012"}
Oct 30 22:19:43.848155 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.848127    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:43.982750 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.982719    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.982950 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.982778    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.982950 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.982814    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.982950 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.982846    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.982950 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.982919    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.982950 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.982949    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.983236 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.982980    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.983236 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.983018    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.983236 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.983056    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.983236 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.983087    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.983236 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.983114    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.983236 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.983212    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:43.983746 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.983714    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:43.984610 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.984584    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.984729 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.984645    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr\") pod \"08103390-6605-487e-b4bb-9e367e7dda1f\" (UID: \"08103390-6605-487e-b4bb-9e367e7dda1f\") "
Oct 30 22:19:43.986824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.985018    5664 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:43.986824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.985046    5664 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08103390-6605-487e-b4bb-9e367e7dda1f-metrics-client-ca\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:43.986824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.985647    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:43.986824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.985875    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:43.986824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.986210    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:43.986824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.986555    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:43.986824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.986694    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:43.987240 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.987000    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:43.987501 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.987471    5664 generic.go:356] "Generic (PLEG): container finished" podID="08103390-6605-487e-b4bb-9e367e7dda1f" containerID="cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476" exitCode=0
Oct 30 22:19:43.987622 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.987589    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerDied","Data":"cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476"}
Oct 30 22:19:43.987682 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.987663    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08103390-6605-487e-b4bb-9e367e7dda1f","Type":"ContainerDied","Data":"3a3dec3c50c3664399b01d817628bf0151f02624a2be2a70e3905315c1338337"}
Oct 30 22:19:43.987733 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.987713    5664 scope.go:122] "RemoveContainer" containerID="af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e"
Oct 30 22:19:43.987794 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.987747    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 22:19:43.989583 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.989555    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:43.989683 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.989611    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "kube-api-access-z2vgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:43.990776 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.990754    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:43.993336 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.993313    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:43.998764 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:43.998743    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config" pod "08103390-6605-487e-b4bb-9e367e7dda1f" (UID: "08103390-6605-487e-b4bb-9e367e7dda1f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:44.005940 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.005918    5664 scope.go:122] "RemoveContainer" containerID="591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11"
Oct 30 22:19:44.013394 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.013374    5664 scope.go:122] "RemoveContainer" containerID="74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351"
Oct 30 22:19:44.020426 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.020406    5664 scope.go:122] "RemoveContainer" containerID="cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476"
Oct 30 22:19:44.027486 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.027462    5664 scope.go:122] "RemoveContainer" containerID="8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef"
Oct 30 22:19:44.034871 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.034853    5664 scope.go:122] "RemoveContainer" containerID="fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012"
Oct 30 22:19:44.041864 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.041844    5664 scope.go:122] "RemoveContainer" containerID="8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e"
Oct 30 22:19:44.048851 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.048834    5664 scope.go:122] "RemoveContainer" containerID="af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e"
Oct 30 22:19:44.049138 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:44.049120    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e\": container with ID starting with af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e not found: ID does not exist" containerID="af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e"
Oct 30 22:19:44.049196 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.049147    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e"} err="failed to get container status \"af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e\": rpc error: code = NotFound desc = could not find container \"af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e\": container with ID starting with af3cb95beb0ff5c86cff042692b4f9025352612b8ed034a5132c94c2d121723e not found: ID does not exist"
Oct 30 22:19:44.049196 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.049180    5664 scope.go:122] "RemoveContainer" containerID="591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11"
Oct 30 22:19:44.049420 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:44.049404    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11\": container with ID starting with 591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11 not found: ID does not exist" containerID="591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11"
Oct 30 22:19:44.049461 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.049423    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11"} err="failed to get container status \"591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11\": rpc error: code = NotFound desc = could not find container \"591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11\": container with ID starting with 591050e0ddb9b5cd63f7c655bbb0960ea34dc9a96b493fbb70a51a1b9181fd11 not found: ID does not exist"
Oct 30 22:19:44.049461 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.049435    5664 scope.go:122] "RemoveContainer" containerID="74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351"
Oct 30 22:19:44.049661 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:44.049647    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351\": container with ID starting with 74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351 not found: ID does not exist" containerID="74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351"
Oct 30 22:19:44.049700 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.049664    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351"} err="failed to get container status \"74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351\": rpc error: code = NotFound desc = could not find container \"74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351\": container with ID starting with 74e3ab88cc36ae56957176ce6d4104dacc568823a27469214656c7e46191a351 not found: ID does not exist"
Oct 30 22:19:44.049700 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.049675    5664 scope.go:122] "RemoveContainer" containerID="cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476"
Oct 30 22:19:44.049922 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:44.049879    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476\": container with ID starting with cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476 not found: ID does not exist" containerID="cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476"
Oct 30 22:19:44.049966 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.049934    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476"} err="failed to get container status \"cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476\": rpc error: code = NotFound desc = could not find container \"cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476\": container with ID starting with cf4aed94e97b0491e8bceb591af02d1816c9168d015cba1f5f185e830ef77476 not found: ID does not exist"
Oct 30 22:19:44.049966 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.049951    5664 scope.go:122] "RemoveContainer" containerID="8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef"
Oct 30 22:19:44.050215 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:44.050198    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef\": container with ID starting with 8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef not found: ID does not exist" containerID="8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef"
Oct 30 22:19:44.050254 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.050224    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef"} err="failed to get container status \"8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef\": rpc error: code = NotFound desc = could not find container \"8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef\": container with ID starting with 8840eebcdb03540308a6f271321f983902210ca526fb026f7190c86eec4ff2ef not found: ID does not exist"
Oct 30 22:19:44.050254 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.050240    5664 scope.go:122] "RemoveContainer" containerID="fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012"
Oct 30 22:19:44.050487 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:44.050465    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012\": container with ID starting with fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012 not found: ID does not exist" containerID="fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012"
Oct 30 22:19:44.050527 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.050491    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012"} err="failed to get container status \"fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012\": rpc error: code = NotFound desc = could not find container \"fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012\": container with ID starting with fd075fcc110e1e125c989bb40d4ada9d762d249dfd10bc101578c8069f9f0012 not found: ID does not exist"
Oct 30 22:19:44.050527 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.050504    5664 scope.go:122] "RemoveContainer" containerID="8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e"
Oct 30 22:19:44.050721 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:44.050705    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e\": container with ID starting with 8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e not found: ID does not exist" containerID="8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e"
Oct 30 22:19:44.050762 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.050725    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e"} err="failed to get container status \"8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e\": rpc error: code = NotFound desc = could not find container \"8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e\": container with ID starting with 8e863e09d5212d79123c36c20274b9aa7d1cdde7446fc51a92ce05556127180e not found: ID does not exist"
Oct 30 22:19:44.086252 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086223    5664 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-config-out\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086252 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086253    5664 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086263    5664 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-cluster-tls-config\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086272    5664 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-tls-assets\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086281    5664 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086290    5664 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-web-config\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086299    5664 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086311    5664 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-config-volume\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086321    5664 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08103390-6605-487e-b4bb-9e367e7dda1f-alertmanager-main-db\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086330    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2vgr\" (UniqueName: \"kubernetes.io/projected/08103390-6605-487e-b4bb-9e367e7dda1f-kube-api-access-z2vgr\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.086413 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.086340    5664 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08103390-6605-487e-b4bb-9e367e7dda1f-secret-alertmanager-main-tls\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:44.313412 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.313378    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:19:44.320686 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:44.320659    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 22:19:45.866356 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:45.866321    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" path="/var/lib/kubelet/pods/08103390-6605-487e-b4bb-9e367e7dda1f/volumes"
Oct 30 22:19:46.610659 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.610629    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:46.610659 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.610671    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.610697    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.610734    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.610762    5664 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.610774    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.610790    5664 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.610847    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:20:02.610827552 +0000 UTC m=+32.862291013 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-config" not registered
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.610864    5664 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.610865    5664 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.610866    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:20:02.610857703 +0000 UTC m=+32.862321149 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 22:19:46.610950 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.610866    5664 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 22:19:46.611257 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.610960    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:20:02.610947462 +0000 UTC m=+32.862410909 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 22:19:46.611257 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.610973    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:20:02.610967317 +0000 UTC m=+32.862430763 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 22:19:46.611257 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.611006    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:20:02.610992521 +0000 UTC m=+32.862455989 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : object "openshift-console"/"service-ca" not registered
Oct 30 22:19:46.711892 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.711866    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"console-688b666f4d-5pmrq\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") " pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:46.712043 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.712028    5664 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 22:19:46.712091 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.712050    5664 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 22:19:46.712091 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.712062    5664 projected.go:196] Error preparing data for projected volume kube-api-access-f78rc for pod openshift-console/console-688b666f4d-5pmrq: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:46.712153 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:46.712109    5664 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc podName:ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28 nodeName:}" failed. No retries permitted until 2026-10-30 22:20:02.712093878 +0000 UTC m=+32.963557324 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-f78rc" (UniqueName: "kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc") pod "console-688b666f4d-5pmrq" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 22:19:46.903639 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.903609    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:19:46.904098 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.903959    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="prometheus" containerID="cri-o://e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229" gracePeriod=600
Oct 30 22:19:46.904203 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.904180    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0" gracePeriod=600
Oct 30 22:19:46.904340 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.904240    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="thanos-sidecar" containerID="cri-o://1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1" gracePeriod=600
Oct 30 22:19:46.904340 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.904268    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy-web" containerID="cri-o://75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b" gracePeriod=600
Oct 30 22:19:46.904340 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.904251    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="config-reloader" containerID="cri-o://b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b" gracePeriod=600
Oct 30 22:19:46.904340 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:46.904239    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy" containerID="cri-o://67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4" gracePeriod=600
Oct 30 22:19:47.146076 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.146054    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:47.316623 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316520    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316623 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316573    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316623 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316593    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316623 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316617    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316987 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316641    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316987 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316659    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316987 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316678    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316987 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316809    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316987 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316892    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316987 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316947    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.316987 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.316975    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317027    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317082    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317115    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317147    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317168    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:47.317317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317178    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:47.317317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317180    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317261    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317301    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317331    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317363    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets\") pod \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\" (UID: \"e92a79fb-ac06-45b2-b606-5fb4eabf94d7\") "
Oct 30 22:19:47.317748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317513    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:47.317923 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317748    5664 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.317923 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317771    5664 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.317923 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.317788    5664 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-db\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.318165 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.318133    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:47.318969 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.318883    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:47.319625 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.319598    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:47.320399 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.320336    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:47.320682 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.320652    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:47.320941 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.320807    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:47.320941 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.320842    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "kube-api-access-pnlcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:47.321078 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.320983    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:47.321245 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.321174    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:47.321245 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.321212    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:47.321533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.321514    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:47.321642 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.321621    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:47.322001 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.321978    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:47.322160 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.322141    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:47.322402 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.322378    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:47.322603 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.322585    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:47.331097 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.331075    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config" pod "e92a79fb-ac06-45b2-b606-5fb4eabf94d7" (UID: "e92a79fb-ac06-45b2-b606-5fb4eabf94d7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:47.418533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418490    5664 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418528    5664 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418540    5664 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-metrics-client-ca\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418549    5664 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-web-config\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418559    5664 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-metrics-client-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418568    5664 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-kube-rbac-proxy\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418577    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pnlcg\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-kube-api-access-pnlcg\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418586    5664 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418596    5664 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418605    5664 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418613    5664 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418622    5664 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418630    5664 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-secret-grpc-tls\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418638    5664 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418647    5664 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config-out\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418655    5664 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-tls-assets\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:47.418774 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:47.418662    5664 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/e92a79fb-ac06-45b2-b606-5fb4eabf94d7-config\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:48.010110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010077    5664 generic.go:356] "Generic (PLEG): container finished" podID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerID="30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0" exitCode=0
Oct 30 22:19:48.010110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010100    5664 generic.go:356] "Generic (PLEG): container finished" podID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerID="67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4" exitCode=0
Oct 30 22:19:48.010110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010107    5664 generic.go:356] "Generic (PLEG): container finished" podID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerID="75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b" exitCode=0
Oct 30 22:19:48.010110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010112    5664 generic.go:356] "Generic (PLEG): container finished" podID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerID="1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1" exitCode=0
Oct 30 22:19:48.010110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010118    5664 generic.go:356] "Generic (PLEG): container finished" podID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerID="b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b" exitCode=0
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010123    5664 generic.go:356] "Generic (PLEG): container finished" podID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerID="e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229" exitCode=0
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010164    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010167    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerDied","Data":"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"}
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010293    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerDied","Data":"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"}
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010335    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerDied","Data":"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"}
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010354    5664 scope.go:122] "RemoveContainer" containerID="30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010375    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerDied","Data":"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"}
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010417    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerDied","Data":"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"}
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010456    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerDied","Data":"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"}
Oct 30 22:19:48.010748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.010498    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"e92a79fb-ac06-45b2-b606-5fb4eabf94d7","Type":"ContainerDied","Data":"7539f30ff407b8642f151d28b324a4fb27c751a44529fb60cc780195b0e4d274"}
Oct 30 22:19:48.017977 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.017953    5664 scope.go:122] "RemoveContainer" containerID="67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"
Oct 30 22:19:48.024494 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.024474    5664 scope.go:122] "RemoveContainer" containerID="75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"
Oct 30 22:19:48.031187 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.031170    5664 scope.go:122] "RemoveContainer" containerID="1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"
Oct 30 22:19:48.039613 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.039588    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:19:48.041077 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.041058    5664 scope.go:122] "RemoveContainer" containerID="b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"
Oct 30 22:19:48.045312 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.045288    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:19:48.048612 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.048586    5664 scope.go:122] "RemoveContainer" containerID="e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"
Oct 30 22:19:48.055846 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.055823    5664 scope.go:122] "RemoveContainer" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"
Oct 30 22:19:48.062232 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.062217    5664 scope.go:122] "RemoveContainer" containerID="30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"
Oct 30 22:19:48.062515 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:48.062494    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": container with ID starting with 30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0 not found: ID does not exist" containerID="30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"
Oct 30 22:19:48.062615 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.062523    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"} err="failed to get container status \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": rpc error: code = NotFound desc = could not find container \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": container with ID starting with 30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0 not found: ID does not exist"
Oct 30 22:19:48.062615 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.062542    5664 scope.go:122] "RemoveContainer" containerID="67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"
Oct 30 22:19:48.062804 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:48.062784    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": container with ID starting with 67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4 not found: ID does not exist" containerID="67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"
Oct 30 22:19:48.062838 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.062812    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"} err="failed to get container status \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": rpc error: code = NotFound desc = could not find container \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": container with ID starting with 67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4 not found: ID does not exist"
Oct 30 22:19:48.062838 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.062829    5664 scope.go:122] "RemoveContainer" containerID="75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"
Oct 30 22:19:48.063087 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:48.063070    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": container with ID starting with 75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b not found: ID does not exist" containerID="75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"
Oct 30 22:19:48.063148 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.063091    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"} err="failed to get container status \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": rpc error: code = NotFound desc = could not find container \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": container with ID starting with 75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b not found: ID does not exist"
Oct 30 22:19:48.063148 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.063105    5664 scope.go:122] "RemoveContainer" containerID="1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"
Oct 30 22:19:48.063326 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:48.063312    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": container with ID starting with 1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1 not found: ID does not exist" containerID="1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"
Oct 30 22:19:48.063366 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.063329    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"} err="failed to get container status \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": rpc error: code = NotFound desc = could not find container \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": container with ID starting with 1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1 not found: ID does not exist"
Oct 30 22:19:48.063366 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.063340    5664 scope.go:122] "RemoveContainer" containerID="b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"
Oct 30 22:19:48.063543 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:48.063528    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": container with ID starting with b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b not found: ID does not exist" containerID="b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"
Oct 30 22:19:48.063576 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.063547    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"} err="failed to get container status \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": rpc error: code = NotFound desc = could not find container \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": container with ID starting with b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b not found: ID does not exist"
Oct 30 22:19:48.063576 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.063558    5664 scope.go:122] "RemoveContainer" containerID="e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"
Oct 30 22:19:48.063786 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:48.063769    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": container with ID starting with e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229 not found: ID does not exist" containerID="e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"
Oct 30 22:19:48.063826 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.063789    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"} err="failed to get container status \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": rpc error: code = NotFound desc = could not find container \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": container with ID starting with e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229 not found: ID does not exist"
Oct 30 22:19:48.063826 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.063802    5664 scope.go:122] "RemoveContainer" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"
Oct 30 22:19:48.064045 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:19:48.064024    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": container with ID starting with 9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc not found: ID does not exist" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"
Oct 30 22:19:48.064085 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.064052    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"} err="failed to get container status \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": rpc error: code = NotFound desc = could not find container \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": container with ID starting with 9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc not found: ID does not exist"
Oct 30 22:19:48.064085 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.064068    5664 scope.go:122] "RemoveContainer" containerID="30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"
Oct 30 22:19:48.064321 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.064303    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"} err="failed to get container status \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": rpc error: code = NotFound desc = could not find container \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": container with ID starting with 30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0 not found: ID does not exist"
Oct 30 22:19:48.064367 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.064322    5664 scope.go:122] "RemoveContainer" containerID="67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"
Oct 30 22:19:48.064551 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.064531    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"} err="failed to get container status \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": rpc error: code = NotFound desc = could not find container \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": container with ID starting with 67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4 not found: ID does not exist"
Oct 30 22:19:48.064627 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.064552    5664 scope.go:122] "RemoveContainer" containerID="75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"
Oct 30 22:19:48.064766 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.064751    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"} err="failed to get container status \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": rpc error: code = NotFound desc = could not find container \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": container with ID starting with 75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b not found: ID does not exist"
Oct 30 22:19:48.064811 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.064766    5664 scope.go:122] "RemoveContainer" containerID="1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"
Oct 30 22:19:48.065005 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.064987    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"} err="failed to get container status \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": rpc error: code = NotFound desc = could not find container \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": container with ID starting with 1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1 not found: ID does not exist"
Oct 30 22:19:48.065055 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.065006    5664 scope.go:122] "RemoveContainer" containerID="b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"
Oct 30 22:19:48.065230 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.065211    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"} err="failed to get container status \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": rpc error: code = NotFound desc = could not find container \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": container with ID starting with b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b not found: ID does not exist"
Oct 30 22:19:48.065230 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.065231    5664 scope.go:122] "RemoveContainer" containerID="e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"
Oct 30 22:19:48.065429 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.065415    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"} err="failed to get container status \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": rpc error: code = NotFound desc = could not find container \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": container with ID starting with e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229 not found: ID does not exist"
Oct 30 22:19:48.065467 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.065429    5664 scope.go:122] "RemoveContainer" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"
Oct 30 22:19:48.065667 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.065643    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"} err="failed to get container status \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": rpc error: code = NotFound desc = could not find container \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": container with ID starting with 9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc not found: ID does not exist"
Oct 30 22:19:48.065667 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.065666    5664 scope.go:122] "RemoveContainer" containerID="30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"
Oct 30 22:19:48.065883 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.065864    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"} err="failed to get container status \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": rpc error: code = NotFound desc = could not find container \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": container with ID starting with 30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0 not found: ID does not exist"
Oct 30 22:19:48.065941 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.065884    5664 scope.go:122] "RemoveContainer" containerID="67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"
Oct 30 22:19:48.066116 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.066099    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"} err="failed to get container status \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": rpc error: code = NotFound desc = could not find container \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": container with ID starting with 67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4 not found: ID does not exist"
Oct 30 22:19:48.066167 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.066116    5664 scope.go:122] "RemoveContainer" containerID="75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"
Oct 30 22:19:48.066336 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.066318    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"} err="failed to get container status \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": rpc error: code = NotFound desc = could not find container \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": container with ID starting with 75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b not found: ID does not exist"
Oct 30 22:19:48.066380 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.066337    5664 scope.go:122] "RemoveContainer" containerID="1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"
Oct 30 22:19:48.066568 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.066552    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"} err="failed to get container status \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": rpc error: code = NotFound desc = could not find container \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": container with ID starting with 1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1 not found: ID does not exist"
Oct 30 22:19:48.066620 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.066569    5664 scope.go:122] "RemoveContainer" containerID="b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"
Oct 30 22:19:48.066830 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.066811    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"} err="failed to get container status \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": rpc error: code = NotFound desc = could not find container \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": container with ID starting with b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b not found: ID does not exist"
Oct 30 22:19:48.066918 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.066831    5664 scope.go:122] "RemoveContainer" containerID="e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"
Oct 30 22:19:48.067070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067054    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"} err="failed to get container status \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": rpc error: code = NotFound desc = could not find container \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": container with ID starting with e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229 not found: ID does not exist"
Oct 30 22:19:48.067117 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067070    5664 scope.go:122] "RemoveContainer" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"
Oct 30 22:19:48.067290 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067274    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"} err="failed to get container status \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": rpc error: code = NotFound desc = could not find container \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": container with ID starting with 9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc not found: ID does not exist"
Oct 30 22:19:48.067329 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067290    5664 scope.go:122] "RemoveContainer" containerID="30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"
Oct 30 22:19:48.067499 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067482    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"} err="failed to get container status \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": rpc error: code = NotFound desc = could not find container \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": container with ID starting with 30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0 not found: ID does not exist"
Oct 30 22:19:48.067540 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067500    5664 scope.go:122] "RemoveContainer" containerID="67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"
Oct 30 22:19:48.067701 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067687    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"} err="failed to get container status \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": rpc error: code = NotFound desc = could not find container \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": container with ID starting with 67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4 not found: ID does not exist"
Oct 30 22:19:48.067740 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067701    5664 scope.go:122] "RemoveContainer" containerID="75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"
Oct 30 22:19:48.067938 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067919    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"} err="failed to get container status \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": rpc error: code = NotFound desc = could not find container \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": container with ID starting with 75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b not found: ID does not exist"
Oct 30 22:19:48.068017 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.067939    5664 scope.go:122] "RemoveContainer" containerID="1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"
Oct 30 22:19:48.068312 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.068293    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"} err="failed to get container status \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": rpc error: code = NotFound desc = could not find container \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": container with ID starting with 1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1 not found: ID does not exist"
Oct 30 22:19:48.068388 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.068313    5664 scope.go:122] "RemoveContainer" containerID="b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"
Oct 30 22:19:48.068570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.068538    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"} err="failed to get container status \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": rpc error: code = NotFound desc = could not find container \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": container with ID starting with b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b not found: ID does not exist"
Oct 30 22:19:48.068617 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.068569    5664 scope.go:122] "RemoveContainer" containerID="e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"
Oct 30 22:19:48.068815 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.068795    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"} err="failed to get container status \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": rpc error: code = NotFound desc = could not find container \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": container with ID starting with e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229 not found: ID does not exist"
Oct 30 22:19:48.068860 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.068824    5664 scope.go:122] "RemoveContainer" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"
Oct 30 22:19:48.069090 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.069071    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"} err="failed to get container status \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": rpc error: code = NotFound desc = could not find container \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": container with ID starting with 9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc not found: ID does not exist"
Oct 30 22:19:48.069170 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.069092    5664 scope.go:122] "RemoveContainer" containerID="30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"
Oct 30 22:19:48.069354 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.069336    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"} err="failed to get container status \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": rpc error: code = NotFound desc = could not find container \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": container with ID starting with 30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0 not found: ID does not exist"
Oct 30 22:19:48.069393 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.069354    5664 scope.go:122] "RemoveContainer" containerID="67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"
Oct 30 22:19:48.069595 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.069578    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"} err="failed to get container status \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": rpc error: code = NotFound desc = could not find container \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": container with ID starting with 67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4 not found: ID does not exist"
Oct 30 22:19:48.069635 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.069597    5664 scope.go:122] "RemoveContainer" containerID="75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"
Oct 30 22:19:48.069821 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.069802    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"} err="failed to get container status \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": rpc error: code = NotFound desc = could not find container \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": container with ID starting with 75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b not found: ID does not exist"
Oct 30 22:19:48.069886 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.069821    5664 scope.go:122] "RemoveContainer" containerID="1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"
Oct 30 22:19:48.070057 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070041    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"} err="failed to get container status \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": rpc error: code = NotFound desc = could not find container \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": container with ID starting with 1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1 not found: ID does not exist"
Oct 30 22:19:48.070110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070057    5664 scope.go:122] "RemoveContainer" containerID="b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"
Oct 30 22:19:48.070276 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070257    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"} err="failed to get container status \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": rpc error: code = NotFound desc = could not find container \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": container with ID starting with b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b not found: ID does not exist"
Oct 30 22:19:48.070345 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070279    5664 scope.go:122] "RemoveContainer" containerID="e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"
Oct 30 22:19:48.070509 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070490    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"} err="failed to get container status \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": rpc error: code = NotFound desc = could not find container \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": container with ID starting with e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229 not found: ID does not exist"
Oct 30 22:19:48.070551 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070510    5664 scope.go:122] "RemoveContainer" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"
Oct 30 22:19:48.070727 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070711    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"} err="failed to get container status \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": rpc error: code = NotFound desc = could not find container \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": container with ID starting with 9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc not found: ID does not exist"
Oct 30 22:19:48.070780 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070727    5664 scope.go:122] "RemoveContainer" containerID="30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"
Oct 30 22:19:48.070933 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070917    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0"} err="failed to get container status \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": rpc error: code = NotFound desc = could not find container \"30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0\": container with ID starting with 30e018ddb9fa1fa90f09d6572f7daec3974c36201d44d157c73c20510dbfedb0 not found: ID does not exist"
Oct 30 22:19:48.070991 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.070933    5664 scope.go:122] "RemoveContainer" containerID="67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"
Oct 30 22:19:48.071125 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071107    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4"} err="failed to get container status \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": rpc error: code = NotFound desc = could not find container \"67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4\": container with ID starting with 67b4b9fbcbb15a1fdfb8929260ee6073df625a3b41e26b5f98fb77bfbbe439e4 not found: ID does not exist"
Oct 30 22:19:48.071184 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071126    5664 scope.go:122] "RemoveContainer" containerID="75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"
Oct 30 22:19:48.071358 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071338    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b"} err="failed to get container status \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": rpc error: code = NotFound desc = could not find container \"75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b\": container with ID starting with 75d040ae25f8266edf439d07f0f5f91356344759ff04b009307cc33f72e9e61b not found: ID does not exist"
Oct 30 22:19:48.071404 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071358    5664 scope.go:122] "RemoveContainer" containerID="1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"
Oct 30 22:19:48.071549 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071530    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1"} err="failed to get container status \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": rpc error: code = NotFound desc = could not find container \"1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1\": container with ID starting with 1b029b3fb39b4338386a0dbff4e8183bc929f5982d70b6710e5b79fa240a9ae1 not found: ID does not exist"
Oct 30 22:19:48.071637 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071550    5664 scope.go:122] "RemoveContainer" containerID="b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"
Oct 30 22:19:48.071741 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071722    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b"} err="failed to get container status \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": rpc error: code = NotFound desc = could not find container \"b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b\": container with ID starting with b58b5e5fd3f8285f5b283c4b19b7580c0cd6e1cfdf55735532024a051743645b not found: ID does not exist"
Oct 30 22:19:48.071777 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071741    5664 scope.go:122] "RemoveContainer" containerID="e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"
Oct 30 22:19:48.071984 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071965    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229"} err="failed to get container status \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": rpc error: code = NotFound desc = could not find container \"e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229\": container with ID starting with e92b8cf2a52595ea7a94564807e1b302793995bba3753328784bfbee0ce2a229 not found: ID does not exist"
Oct 30 22:19:48.072031 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.071985    5664 scope.go:122] "RemoveContainer" containerID="9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"
Oct 30 22:19:48.072223 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.072204    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc"} err="failed to get container status \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": rpc error: code = NotFound desc = could not find container \"9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc\": container with ID starting with 9032576087cb59a2efa5d45786aa96bac63e1088bc082a5c19040400d37851cc not found: ID does not exist"
Oct 30 22:19:48.076991 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.076970    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:19:48.077247 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077232    5664 cpu_manager.go:414] "containerMap: removing container" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="prometheus"
Oct 30 22:19:48.077311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077250    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="prometheus"
Oct 30 22:19:48.077311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077265    5664 cpu_manager.go:414] "containerMap: removing container" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="alertmanager"
Oct 30 22:19:48.077311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077273    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="alertmanager"
Oct 30 22:19:48.077311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077289    5664 cpu_manager.go:414] "containerMap: removing container" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy"
Oct 30 22:19:48.077311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077296    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy"
Oct 30 22:19:48.077311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077309    5664 cpu_manager.go:414] "containerMap: removing container" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077316    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077337    5664 cpu_manager.go:414] "containerMap: removing container" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy-web"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077345    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy-web"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077354    5664 cpu_manager.go:414] "containerMap: removing container" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy-thanos"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077361    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy-thanos"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077370    5664 cpu_manager.go:414] "containerMap: removing container" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy-metric"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077377    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy-metric"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077389    5664 cpu_manager.go:414] "containerMap: removing container" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077396    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077407    5664 cpu_manager.go:414] "containerMap: removing container" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="thanos-sidecar"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077413    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="thanos-sidecar"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077424    5664 cpu_manager.go:414] "containerMap: removing container" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="prom-label-proxy"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077432    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="prom-label-proxy"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077442    5664 cpu_manager.go:414] "containerMap: removing container" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077448    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077461    5664 cpu_manager.go:414] "containerMap: removing container" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="init-config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077469    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="init-config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077481    5664 cpu_manager.go:414] "containerMap: removing container" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="init-config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077488    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="init-config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077500    5664 cpu_manager.go:414] "containerMap: removing container" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy-web"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077507    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy-web"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077564    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077576    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="thanos-sidecar"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077586    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="prom-label-proxy"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077593    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077602    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="init-config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077609    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="init-config-reloader"
Oct 30 22:19:48.077605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077615    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy-web"
Oct 30 22:19:48.078891 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077623    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="prometheus"
Oct 30 22:19:48.078891 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077630    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="alertmanager"
Oct 30 22:19:48.078891 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077638    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy"
Oct 30 22:19:48.078891 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077646    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="config-reloader"
Oct 30 22:19:48.078891 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077656    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy-web"
Oct 30 22:19:48.078891 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077663    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" containerName="kube-rbac-proxy-thanos"
Oct 30 22:19:48.078891 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.077672    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="08103390-6605-487e-b4bb-9e367e7dda1f" containerName="kube-rbac-proxy-metric"
Oct 30 22:19:48.081460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.081441    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087027    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087031    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087056    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4f76b\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087058    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087058    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087339    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-caus6766sf7av\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087355    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087380    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087421    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087439    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Oct 30 22:19:48.087496 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087451    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Oct 30 22:19:48.088270 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087591    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Oct 30 22:19:48.088270 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087595    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Oct 30 22:19:48.088270 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087600    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Oct 30 22:19:48.088270 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.087807    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 22:19:48.090754 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.090735    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Oct 30 22:19:48.091351 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.091334    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Oct 30 22:19:48.102397 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.102367    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:19:48.226360 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226328    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a2bedfe-3af0-427a-b3c2-3969f6660845-config-out\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226528 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226379    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226528 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226475    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226528 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226516    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226631 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226598    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226663 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226628    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-web-config\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226663 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226645    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226717 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226703    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226732    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226780 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226765    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226822 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226811    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226853 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226829    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pflp\" (UniqueName: \"kubernetes.io/projected/5a2bedfe-3af0-427a-b3c2-3969f6660845-kube-api-access-2pflp\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226918    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226986 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226955    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.226986 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.226972    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-config\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.227051 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.227005    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.227091 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.227052    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.227176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.227155    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.227234 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.227201    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.227307 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.227292    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a2bedfe-3af0-427a-b3c2-3969f6660845-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.328593 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328556    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.328593 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328596    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a2bedfe-3af0-427a-b3c2-3969f6660845-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.328846 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328613    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a2bedfe-3af0-427a-b3c2-3969f6660845-config-out\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.328846 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328757    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.328846 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328785    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.328846 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328802    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.328846 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328834    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329138 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328882    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-web-config\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329138 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328940    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329138 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328961    5664 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 22:19:48.329138 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.328989    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329138 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329017    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329138 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329041    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329138 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329070    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329138 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329095    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pflp\" (UniqueName: \"kubernetes.io/projected/5a2bedfe-3af0-427a-b3c2-3969f6660845-kube-api-access-2pflp\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329141    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329172    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329198    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-config\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329228    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329282    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329342    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329340    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.329525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.329498    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.330061 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.330026    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.330304 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.330234    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.331517 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.331175    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.333183 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.332192    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.333183 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.332547    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5a2bedfe-3af0-427a-b3c2-3969f6660845-config-out\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.333183 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.332620    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5a2bedfe-3af0-427a-b3c2-3969f6660845-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.333183 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.333047    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.333734 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.333490    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.333734 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.333622    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.334080 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.334002    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a2bedfe-3af0-427a-b3c2-3969f6660845-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.334174 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.334093    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.334261 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.334243    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.334354 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.334329    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-config\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.334451 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.334336    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.334792 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.334772    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.335777 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.335756    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.336536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.336513    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5a2bedfe-3af0-427a-b3c2-3969f6660845-web-config\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.346116 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.346097    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pflp\" (UniqueName: \"kubernetes.io/projected/5a2bedfe-3af0-427a-b3c2-3969f6660845-kube-api-access-2pflp\") pod \"prometheus-k8s-0\" (UID: \"5a2bedfe-3af0-427a-b3c2-3969f6660845\") " pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.397674 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.397640    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:48.685502 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:48.685474    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Oct 30 22:19:48.688889 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:19:48.688861    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2bedfe_3af0_427a_b3c2_3969f6660845.slice/crio-2e1b4f7fe3062e2a5c89bbf73ffbf1d9ae6f566124004758d8f17be52faf9fb7 WatchSource:0}: Error finding container 2e1b4f7fe3062e2a5c89bbf73ffbf1d9ae6f566124004758d8f17be52faf9fb7: Status 404 returned error can't find the container with id 2e1b4f7fe3062e2a5c89bbf73ffbf1d9ae6f566124004758d8f17be52faf9fb7
Oct 30 22:19:49.016142 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:49.016068    5664 generic.go:356] "Generic (PLEG): container finished" podID="5a2bedfe-3af0-427a-b3c2-3969f6660845" containerID="f3f46e2072f3c9aa0dc2fd197a7c154fb506feb053962356c2910e29178b960c" exitCode=0
Oct 30 22:19:49.016142 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:49.016126    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2bedfe-3af0-427a-b3c2-3969f6660845","Type":"ContainerDied","Data":"f3f46e2072f3c9aa0dc2fd197a7c154fb506feb053962356c2910e29178b960c"}
Oct 30 22:19:49.016511 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:49.016176    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2bedfe-3af0-427a-b3c2-3969f6660845","Type":"ContainerStarted","Data":"2e1b4f7fe3062e2a5c89bbf73ffbf1d9ae6f566124004758d8f17be52faf9fb7"}
Oct 30 22:19:49.867658 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:49.867621    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e92a79fb-ac06-45b2-b606-5fb4eabf94d7" path="/var/lib/kubelet/pods/e92a79fb-ac06-45b2-b606-5fb4eabf94d7/volumes"
Oct 30 22:19:50.025253 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:50.025221    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2bedfe-3af0-427a-b3c2-3969f6660845","Type":"ContainerStarted","Data":"4d9cb910ad5cb77247cf32095191a4bc90ec6d9e16383cbae69ec6553b76464d"}
Oct 30 22:19:50.025629 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:50.025284    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2bedfe-3af0-427a-b3c2-3969f6660845","Type":"ContainerStarted","Data":"41ed36943a8efb0f5b2361ba881bc941998b9ee9c7ad42909d23a6ece870dc05"}
Oct 30 22:19:50.025629 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:50.025310    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2bedfe-3af0-427a-b3c2-3969f6660845","Type":"ContainerStarted","Data":"888ce240fa88b74f6f6df9758aebd81ae437265a87b4689f35e09ac6ec4bb2cd"}
Oct 30 22:19:50.025629 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:50.025333    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2bedfe-3af0-427a-b3c2-3969f6660845","Type":"ContainerStarted","Data":"207bd6b6a5845a1ffbb936e551c166365b956942eed14b9e533f4fe7e8cd9bf3"}
Oct 30 22:19:50.025629 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:50.025357    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2bedfe-3af0-427a-b3c2-3969f6660845","Type":"ContainerStarted","Data":"5ec3c948da080afed50074cb6806c43d4129fae4d73e68784a19a1522dce92ad"}
Oct 30 22:19:50.025629 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:50.025378    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5a2bedfe-3af0-427a-b3c2-3969f6660845","Type":"ContainerStarted","Data":"01f6f170875905d606fa6f43a303204cf2bcbeb14cd9d16185cb216d7bd99ce6"}
Oct 30 22:19:50.062881 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:50.062813    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.047276175 podStartE2EDuration="2.062801401s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.525226ms" isStatefulPod=false podCreationTimestamp="2026-10-30 22:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:19:50.061674201 +0000 UTC m=+20.313137670" watchObservedRunningTime="2026-10-30 22:19:50.062801401 +0000 UTC m=+20.314264863"
Oct 30 22:19:52.624380 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:52.624352    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ptpfs"
Oct 30 22:19:53.398795 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:53.398758    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:19:53.821177 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:53.821098    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hdm75"
Oct 30 22:19:55.859664 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:55.859625    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-688b666f4d-5pmrq" podUID="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" containerName="console" containerID="cri-o://7e988ab5297b90484d1859c2681f17a7f44415615e746a0b8e44435594935283" gracePeriod=15
Oct 30 22:19:56.049845 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.049809    5664 generic.go:356] "Generic (PLEG): container finished" podID="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" containerID="7e988ab5297b90484d1859c2681f17a7f44415615e746a0b8e44435594935283" exitCode=0
Oct 30 22:19:56.050104 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.049916    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688b666f4d-5pmrq" event={"ID":"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28","Type":"ContainerDied","Data":"7e988ab5297b90484d1859c2681f17a7f44415615e746a0b8e44435594935283"}
Oct 30 22:19:56.106460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.106435    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:56.196571 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.196542    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") pod \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") "
Oct 30 22:19:56.196767 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.196607    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") pod \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") "
Oct 30 22:19:56.196767 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.196636    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") pod \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") "
Oct 30 22:19:56.196767 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.196707    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") pod \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") "
Oct 30 22:19:56.196767 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.196737    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") pod \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") "
Oct 30 22:19:56.196985 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.196776    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp\") pod \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") "
Oct 30 22:19:56.196985 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.196825    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") pod \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\" (UID: \"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28\") "
Oct 30 22:19:56.197154 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.197122    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp" pod "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:19:56.197217 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.197152    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config" pod "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:56.197217 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.197164    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert" pod "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:56.197217 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.197182    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca" pod "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:19:56.199130 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.199098    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config" pod "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:56.199242 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.199165    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert" pod "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:19:56.199242 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.199180    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc" pod "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" (UID: "ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28"). InnerVolumeSpecName "kube-api-access-f78rc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:19:56.298349 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.298321    5664 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-service-ca\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:56.298349 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.298349    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f78rc\" (UniqueName: \"kubernetes.io/projected/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-kube-api-access-f78rc\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:56.298533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.298361    5664 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-tmp\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:56.298533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.298370    5664 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-serving-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:56.298533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.298378    5664 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-oauth-config\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:56.298533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.298388    5664 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-console-config\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:56.298533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:56.298396    5664 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28-oauth-serving-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:19:57.054208 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:57.054181    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688b666f4d-5pmrq"
Oct 30 22:19:57.054208 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:57.054187    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688b666f4d-5pmrq" event={"ID":"ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28","Type":"ContainerDied","Data":"fcdb921f5a039678d98a0e65e7a92593630132e11262305e0b58767321407e56"}
Oct 30 22:19:57.054710 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:57.054239    5664 scope.go:122] "RemoveContainer" containerID="7e988ab5297b90484d1859c2681f17a7f44415615e746a0b8e44435594935283"
Oct 30 22:19:57.077702 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:57.077676    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688b666f4d-5pmrq"]
Oct 30 22:19:57.088119 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:57.088091    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-688b666f4d-5pmrq"]
Oct 30 22:19:57.866946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:57.866912    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" path="/var/lib/kubelet/pods/ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28/volumes"
Oct 30 22:19:59.258364 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:19:59.258332    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-27nh7"
Oct 30 22:20:07.515357 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:20:07.515311    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:20:35.384317 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:20:35.384264    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:20:35.399962 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:20:35.399937    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:20:36.206847 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:20:36.206821    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Oct 30 22:20:37.514707 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:20:37.514672    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:21:07.515332 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:21:07.515291    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:21:37.514239 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:21:37.514199    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:22:07.515609 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:22:07.515572    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:22:37.514122 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:22:37.514078    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:23:07.514474 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:23:07.514428    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:23:37.514372 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:23:37.514334    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:24:07.515435 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:24:07.515396    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:24:14.718610 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.718514    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"]
Oct 30 22:24:14.719083 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.718821    5664 cpu_manager.go:414] "containerMap: removing container" podUID="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" containerName="console"
Oct 30 22:24:14.719083 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.718833    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" containerName="console"
Oct 30 22:24:14.719083 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.718884    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca5a6fe7-f3f9-46eb-a4f5-f113c89e0c28" containerName="console"
Oct 30 22:24:14.721943 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.721921    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:14.733546 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.733526    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 22:24:14.733637 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.733554    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 22:24:14.733759 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.733747    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sglkh\""
Oct 30 22:24:14.743405 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.743382    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"]
Oct 30 22:24:14.784163 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.784135    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx4vd\" (UniqueName: \"kubernetes.io/projected/1bdd85b5-8add-4704-83f1-1992e2dfed5a-kube-api-access-hx4vd\") pod \"cert-manager-webhook-5dbbbf75cc-865tq\" (UID: \"1bdd85b5-8add-4704-83f1-1992e2dfed5a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:14.784268 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.784186    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bdd85b5-8add-4704-83f1-1992e2dfed5a-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-865tq\" (UID: \"1bdd85b5-8add-4704-83f1-1992e2dfed5a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:14.885519 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.885492    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bdd85b5-8add-4704-83f1-1992e2dfed5a-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-865tq\" (UID: \"1bdd85b5-8add-4704-83f1-1992e2dfed5a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:14.885683 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.885546    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx4vd\" (UniqueName: \"kubernetes.io/projected/1bdd85b5-8add-4704-83f1-1992e2dfed5a-kube-api-access-hx4vd\") pod \"cert-manager-webhook-5dbbbf75cc-865tq\" (UID: \"1bdd85b5-8add-4704-83f1-1992e2dfed5a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:14.906398 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.906372    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx4vd\" (UniqueName: \"kubernetes.io/projected/1bdd85b5-8add-4704-83f1-1992e2dfed5a-kube-api-access-hx4vd\") pod \"cert-manager-webhook-5dbbbf75cc-865tq\" (UID: \"1bdd85b5-8add-4704-83f1-1992e2dfed5a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:14.911463 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:14.911443    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1bdd85b5-8add-4704-83f1-1992e2dfed5a-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-865tq\" (UID: \"1bdd85b5-8add-4704-83f1-1992e2dfed5a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:15.039871 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:15.039795    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:15.313519 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:15.313435    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"]
Oct 30 22:24:15.316924 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:24:15.316885    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1bdd85b5_8add_4704_83f1_1992e2dfed5a.slice/crio-cc53c79c02b8241a03684ebd2625fba03c4bbe7ceedaf2970ff37d29ccdf9069 WatchSource:0}: Error finding container cc53c79c02b8241a03684ebd2625fba03c4bbe7ceedaf2970ff37d29ccdf9069: Status 404 returned error can't find the container with id cc53c79c02b8241a03684ebd2625fba03c4bbe7ceedaf2970ff37d29ccdf9069
Oct 30 22:24:15.318872 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:15.318853    5664 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:24:15.882882 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:15.882836    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq" event={"ID":"1bdd85b5-8add-4704-83f1-1992e2dfed5a","Type":"ContainerStarted","Data":"cc53c79c02b8241a03684ebd2625fba03c4bbe7ceedaf2970ff37d29ccdf9069"}
Oct 30 22:24:18.186717 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.186688    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"]
Oct 30 22:24:18.190166 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.190142    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"
Oct 30 22:24:18.195051 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.195029    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8z8g8\""
Oct 30 22:24:18.199100 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.198131    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"]
Oct 30 22:24:18.209742 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.209713    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv5jl\" (UniqueName: \"kubernetes.io/projected/12c26cd1-c936-429c-921f-8760e9518def-kube-api-access-mv5jl\") pod \"cert-manager-cainjector-5c57bbd4fc-8bzw4\" (UID: \"12c26cd1-c936-429c-921f-8760e9518def\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"
Oct 30 22:24:18.209859 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.209763    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12c26cd1-c936-429c-921f-8760e9518def-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-8bzw4\" (UID: \"12c26cd1-c936-429c-921f-8760e9518def\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"
Oct 30 22:24:18.310188 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.310147    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv5jl\" (UniqueName: \"kubernetes.io/projected/12c26cd1-c936-429c-921f-8760e9518def-kube-api-access-mv5jl\") pod \"cert-manager-cainjector-5c57bbd4fc-8bzw4\" (UID: \"12c26cd1-c936-429c-921f-8760e9518def\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"
Oct 30 22:24:18.310188 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.310196    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12c26cd1-c936-429c-921f-8760e9518def-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-8bzw4\" (UID: \"12c26cd1-c936-429c-921f-8760e9518def\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"
Oct 30 22:24:18.323994 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.323966    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12c26cd1-c936-429c-921f-8760e9518def-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-8bzw4\" (UID: \"12c26cd1-c936-429c-921f-8760e9518def\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"
Oct 30 22:24:18.326991 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.326963    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv5jl\" (UniqueName: \"kubernetes.io/projected/12c26cd1-c936-429c-921f-8760e9518def-kube-api-access-mv5jl\") pod \"cert-manager-cainjector-5c57bbd4fc-8bzw4\" (UID: \"12c26cd1-c936-429c-921f-8760e9518def\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"
Oct 30 22:24:18.504733 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.504650    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"
Oct 30 22:24:18.785314 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.785230    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4"]
Oct 30 22:24:18.789112 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:24:18.789080    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12c26cd1_c936_429c_921f_8760e9518def.slice/crio-989c94f264f5d8a66811231cb1485d29f90cce41127eaabc53d1f19722acc3d8 WatchSource:0}: Error finding container 989c94f264f5d8a66811231cb1485d29f90cce41127eaabc53d1f19722acc3d8: Status 404 returned error can't find the container with id 989c94f264f5d8a66811231cb1485d29f90cce41127eaabc53d1f19722acc3d8
Oct 30 22:24:18.894303 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:18.894269    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4" event={"ID":"12c26cd1-c936-429c-921f-8760e9518def","Type":"ContainerStarted","Data":"989c94f264f5d8a66811231cb1485d29f90cce41127eaabc53d1f19722acc3d8"}
Oct 30 22:24:19.899400 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:19.899369    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq" event={"ID":"1bdd85b5-8add-4704-83f1-1992e2dfed5a","Type":"ContainerStarted","Data":"aae0fa6da65ca0a6ad08893634284e9f9f5f21c3b07457d21edb4a04c7bf9a55"}
Oct 30 22:24:19.899848 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:19.899540    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:19.920503 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:19.920451    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq" podStartSLOduration=2.341575471 podStartE2EDuration="5.920434224s" totalImagesPullingTime="3.578858753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:19.920009759 +0000 UTC m=+290.171473241" watchObservedRunningTime="2026-10-30 22:24:19.920434224 +0000 UTC m=+290.171897692"
Oct 30 22:24:20.903792 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:20.903752    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4" event={"ID":"12c26cd1-c936-429c-921f-8760e9518def","Type":"ContainerStarted","Data":"fc37c04074e721bddeac3be718063c72a5688ae2320c86e696e99e90eebcf6c5"}
Oct 30 22:24:25.905890 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:25.905859    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-865tq"
Oct 30 22:24:25.927337 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:25.927287    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-8bzw4" podStartSLOduration=6.863355773 podStartE2EDuration="7.927273202s" totalImagesPullingTime="1.063917429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:20.925219887 +0000 UTC m=+291.176683357" watchObservedRunningTime="2026-10-30 22:24:25.927273202 +0000 UTC m=+296.178736672"
Oct 30 22:24:32.291255 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.291220    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"]
Oct 30 22:24:32.294504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.294484    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"
Oct 30 22:24:32.300831 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.300702    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-tsrsb\""
Oct 30 22:24:32.300831 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.300789    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 30 22:24:32.300831 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.300813    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 30 22:24:32.304697 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.304671    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"]
Oct 30 22:24:32.324773 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.324743    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c2b870c-91b9-4529-829c-08bd004cd85a-tmp\") pod \"openshift-lws-operator-8465994787-n5xg6\" (UID: \"2c2b870c-91b9-4529-829c-08bd004cd85a\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"
Oct 30 22:24:32.324937 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.324857    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddh6l\" (UniqueName: \"kubernetes.io/projected/2c2b870c-91b9-4529-829c-08bd004cd85a-kube-api-access-ddh6l\") pod \"openshift-lws-operator-8465994787-n5xg6\" (UID: \"2c2b870c-91b9-4529-829c-08bd004cd85a\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"
Oct 30 22:24:32.426280 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.426231    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddh6l\" (UniqueName: \"kubernetes.io/projected/2c2b870c-91b9-4529-829c-08bd004cd85a-kube-api-access-ddh6l\") pod \"openshift-lws-operator-8465994787-n5xg6\" (UID: \"2c2b870c-91b9-4529-829c-08bd004cd85a\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"
Oct 30 22:24:32.426450 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.426293    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c2b870c-91b9-4529-829c-08bd004cd85a-tmp\") pod \"openshift-lws-operator-8465994787-n5xg6\" (UID: \"2c2b870c-91b9-4529-829c-08bd004cd85a\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"
Oct 30 22:24:32.426670 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.426652    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c2b870c-91b9-4529-829c-08bd004cd85a-tmp\") pod \"openshift-lws-operator-8465994787-n5xg6\" (UID: \"2c2b870c-91b9-4529-829c-08bd004cd85a\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"
Oct 30 22:24:32.441723 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.441696    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddh6l\" (UniqueName: \"kubernetes.io/projected/2c2b870c-91b9-4529-829c-08bd004cd85a-kube-api-access-ddh6l\") pod \"openshift-lws-operator-8465994787-n5xg6\" (UID: \"2c2b870c-91b9-4529-829c-08bd004cd85a\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"
Oct 30 22:24:32.605807 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.605713    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"
Oct 30 22:24:32.877853 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.877770    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6"]
Oct 30 22:24:32.881228 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:24:32.881199    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c2b870c_91b9_4529_829c_08bd004cd85a.slice/crio-1847f8dc212b788e0a989080d78d13b02d970cc2c569e645ca6bd008d0c0bcb7 WatchSource:0}: Error finding container 1847f8dc212b788e0a989080d78d13b02d970cc2c569e645ca6bd008d0c0bcb7: Status 404 returned error can't find the container with id 1847f8dc212b788e0a989080d78d13b02d970cc2c569e645ca6bd008d0c0bcb7
Oct 30 22:24:32.945709 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:32.945669    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6" event={"ID":"2c2b870c-91b9-4529-829c-08bd004cd85a","Type":"ContainerStarted","Data":"1847f8dc212b788e0a989080d78d13b02d970cc2c569e645ca6bd008d0c0bcb7"}
Oct 30 22:24:36.962062 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:36.962024    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6" event={"ID":"2c2b870c-91b9-4529-829c-08bd004cd85a","Type":"ContainerStarted","Data":"9e20e938e26f7d508475af89354d104c0d71959d1a2f25407ec77607731cef12"}
Oct 30 22:24:36.982378 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:36.982327    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-8465994787-n5xg6" podStartSLOduration=1.83153772 podStartE2EDuration="4.98231421s" totalImagesPullingTime="3.15077649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:24:36.981666844 +0000 UTC m=+307.233130312" watchObservedRunningTime="2026-10-30 22:24:36.98231421 +0000 UTC m=+307.233777678"
Oct 30 22:24:37.515116 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:24:37.515081    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:24:57.664616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.664561    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"]
Oct 30 22:24:57.668801 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.668782    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.679251 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.679233    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 30 22:24:57.683028 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.683003    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-t9fjn\""
Oct 30 22:24:57.683028 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.683004    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 30 22:24:57.683182 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.683046    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 30 22:24:57.685225 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.685204    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"]
Oct 30 22:24:57.819938 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.819885    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a321c9a-4218-45a7-a73a-83836840736b-metrics-cert\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.820139 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.819955    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a321c9a-4218-45a7-a73a-83836840736b-manager-config\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.820139 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.820071    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a321c9a-4218-45a7-a73a-83836840736b-cert\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.820139 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.820110    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt47v\" (UniqueName: \"kubernetes.io/projected/7a321c9a-4218-45a7-a73a-83836840736b-kube-api-access-vt47v\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.920844 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.920771    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a321c9a-4218-45a7-a73a-83836840736b-manager-config\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.920844 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.920813    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a321c9a-4218-45a7-a73a-83836840736b-cert\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.920844 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.920831    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt47v\" (UniqueName: \"kubernetes.io/projected/7a321c9a-4218-45a7-a73a-83836840736b-kube-api-access-vt47v\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.921072 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.920866    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a321c9a-4218-45a7-a73a-83836840736b-metrics-cert\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.921519 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.921484    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a321c9a-4218-45a7-a73a-83836840736b-manager-config\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.923424 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.923402    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a321c9a-4218-45a7-a73a-83836840736b-metrics-cert\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.923528 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.923453    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a321c9a-4218-45a7-a73a-83836840736b-cert\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.934027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.934003    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt47v\" (UniqueName: \"kubernetes.io/projected/7a321c9a-4218-45a7-a73a-83836840736b-kube-api-access-vt47v\") pod \"lws-controller-manager-6667976c75-8t7zf\" (UID: \"7a321c9a-4218-45a7-a73a-83836840736b\") " pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:57.980012 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:57.979979    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:24:58.254918 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:58.254813    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"]
Oct 30 22:24:58.257785 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:24:58.257757    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a321c9a_4218_45a7_a73a_83836840736b.slice/crio-e874bd08532d7e5d2e9cf9373b19566a1a73c5416600893c1e39b7de648fc549 WatchSource:0}: Error finding container e874bd08532d7e5d2e9cf9373b19566a1a73c5416600893c1e39b7de648fc549: Status 404 returned error can't find the container with id e874bd08532d7e5d2e9cf9373b19566a1a73c5416600893c1e39b7de648fc549
Oct 30 22:24:59.037409 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:24:59.037378    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf" event={"ID":"7a321c9a-4218-45a7-a73a-83836840736b","Type":"ContainerStarted","Data":"e874bd08532d7e5d2e9cf9373b19566a1a73c5416600893c1e39b7de648fc549"}
Oct 30 22:25:01.048308 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:01.048274    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf" event={"ID":"7a321c9a-4218-45a7-a73a-83836840736b","Type":"ContainerStarted","Data":"5c2fe6d6cecd97d3a321a1dbcd59651c29da9481774e26dca70d51616aeee69c"}
Oct 30 22:25:01.048754 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:01.048337    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:25:01.072531 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:01.072482    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf" podStartSLOduration=2.013285614 podStartE2EDuration="4.072469167s" totalImagesPullingTime="2.059183553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:01.070769736 +0000 UTC m=+331.322233203" watchObservedRunningTime="2026-10-30 22:25:01.072469167 +0000 UTC m=+331.323932635"
Oct 30 22:25:07.514751 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:25:07.514715    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:25:12.056834 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:12.056805    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6667976c75-8t7zf"
Oct 30 22:25:32.423409 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.423371    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"]
Oct 30 22:25:32.427427 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.427399    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"
Oct 30 22:25:32.434668 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.434614    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 22:25:32.434771 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.434741    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 22:25:32.434849 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.434818    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-gk4zb\""
Oct 30 22:25:32.446750 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.446720    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"]
Oct 30 22:25:32.483498 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.483463    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxsfr\" (UniqueName: \"kubernetes.io/projected/794a29cb-9298-4fd8-b2a8-ac2acdbffae8-kube-api-access-kxsfr\") pod \"limitador-operator-controller-manager-69b7bb8c5c-fsdcp\" (UID: \"794a29cb-9298-4fd8-b2a8-ac2acdbffae8\") " pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"
Oct 30 22:25:32.583993 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.583956    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxsfr\" (UniqueName: \"kubernetes.io/projected/794a29cb-9298-4fd8-b2a8-ac2acdbffae8-kube-api-access-kxsfr\") pod \"limitador-operator-controller-manager-69b7bb8c5c-fsdcp\" (UID: \"794a29cb-9298-4fd8-b2a8-ac2acdbffae8\") " pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"
Oct 30 22:25:32.604945 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.604913    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxsfr\" (UniqueName: \"kubernetes.io/projected/794a29cb-9298-4fd8-b2a8-ac2acdbffae8-kube-api-access-kxsfr\") pod \"limitador-operator-controller-manager-69b7bb8c5c-fsdcp\" (UID: \"794a29cb-9298-4fd8-b2a8-ac2acdbffae8\") " pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"
Oct 30 22:25:32.740716 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:32.740628    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"
Oct 30 22:25:33.023712 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:33.023637    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"]
Oct 30 22:25:33.026773 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:25:33.026737    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794a29cb_9298_4fd8_b2a8_ac2acdbffae8.slice/crio-d54a3b6f515b763ab5b2ff567dafab8f0b48336227b6168151b4f8ed30c4e091 WatchSource:0}: Error finding container d54a3b6f515b763ab5b2ff567dafab8f0b48336227b6168151b4f8ed30c4e091: Status 404 returned error can't find the container with id d54a3b6f515b763ab5b2ff567dafab8f0b48336227b6168151b4f8ed30c4e091
Oct 30 22:25:33.193662 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:33.193622    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp" event={"ID":"794a29cb-9298-4fd8-b2a8-ac2acdbffae8","Type":"ContainerStarted","Data":"d54a3b6f515b763ab5b2ff567dafab8f0b48336227b6168151b4f8ed30c4e091"}
Oct 30 22:25:36.206486 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:36.206405    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp" event={"ID":"794a29cb-9298-4fd8-b2a8-ac2acdbffae8","Type":"ContainerStarted","Data":"1e233fa0d72a0256f5a1f0614fa1979f649c360e82aa11f5556a6408fb3613fe"}
Oct 30 22:25:36.207071 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:36.206634    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"
Oct 30 22:25:36.229760 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:36.229710    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp" podStartSLOduration=1.312226185 podStartE2EDuration="4.229697646s" totalImagesPullingTime="2.917471461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:25:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:25:36.228298926 +0000 UTC m=+366.479762393" watchObservedRunningTime="2026-10-30 22:25:36.229697646 +0000 UTC m=+366.481161114"
Oct 30 22:25:37.514494 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:25:37.514456    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:25:47.212447 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:25:47.212368    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-69b7bb8c5c-fsdcp"
Oct 30 22:26:07.514398 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:26:07.514361    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:26:37.514940 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:26:37.514889    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:26:40.647713 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.647677    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"]
Oct 30 22:26:40.650950 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.650926    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:26:40.656539 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.656520    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-b87l4\""
Oct 30 22:26:40.660118 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.660096    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"]
Oct 30 22:26:40.680517 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.680479    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 22:26:40.681363 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.681344    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"]
Oct 30 22:26:40.734259 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.734222    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nk8mt\" (UniqueName: \"kubernetes.io/projected/fed16662-e444-4db8-9be6-80fef392612e-kube-api-access-nk8mt\") pod \"limitador-limitador-7c9b87d574-ml7lt\" (UID: \"fed16662-e444-4db8-9be6-80fef392612e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:26:40.734409 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.734278    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fed16662-e444-4db8-9be6-80fef392612e-config-file\") pod \"limitador-limitador-7c9b87d574-ml7lt\" (UID: \"fed16662-e444-4db8-9be6-80fef392612e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:26:40.835194 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.835149    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nk8mt\" (UniqueName: \"kubernetes.io/projected/fed16662-e444-4db8-9be6-80fef392612e-kube-api-access-nk8mt\") pod \"limitador-limitador-7c9b87d574-ml7lt\" (UID: \"fed16662-e444-4db8-9be6-80fef392612e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:26:40.835194 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.835202    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fed16662-e444-4db8-9be6-80fef392612e-config-file\") pod \"limitador-limitador-7c9b87d574-ml7lt\" (UID: \"fed16662-e444-4db8-9be6-80fef392612e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:26:40.835775 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.835755    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/fed16662-e444-4db8-9be6-80fef392612e-config-file\") pod \"limitador-limitador-7c9b87d574-ml7lt\" (UID: \"fed16662-e444-4db8-9be6-80fef392612e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:26:40.848137 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.848104    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nk8mt\" (UniqueName: \"kubernetes.io/projected/fed16662-e444-4db8-9be6-80fef392612e-kube-api-access-nk8mt\") pod \"limitador-limitador-7c9b87d574-ml7lt\" (UID: \"fed16662-e444-4db8-9be6-80fef392612e\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:26:40.966115 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:40.966029    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:26:41.258686 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:41.258606    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"]
Oct 30 22:26:41.261910 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:26:41.261868    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed16662_e444_4db8_9be6_80fef392612e.slice/crio-3d19f586ddb4902d2b1ae1fbf97cf619fbd86194c135896ee6d4ac73f603199f WatchSource:0}: Error finding container 3d19f586ddb4902d2b1ae1fbf97cf619fbd86194c135896ee6d4ac73f603199f: Status 404 returned error can't find the container with id 3d19f586ddb4902d2b1ae1fbf97cf619fbd86194c135896ee6d4ac73f603199f
Oct 30 22:26:41.442406 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:41.442358    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt" event={"ID":"fed16662-e444-4db8-9be6-80fef392612e","Type":"ContainerStarted","Data":"3d19f586ddb4902d2b1ae1fbf97cf619fbd86194c135896ee6d4ac73f603199f"}
Oct 30 22:26:43.450679 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:43.450643    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt" event={"ID":"fed16662-e444-4db8-9be6-80fef392612e","Type":"ContainerStarted","Data":"dc80e05f88e86f4f7e143a0945c9c13e73357c3d777623910d97b1e04e56c871"}
Oct 30 22:26:43.451251 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:43.450771    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:26:43.471690 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:43.471636    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt" podStartSLOduration=2.245913374 podStartE2EDuration="3.471621286s" totalImagesPullingTime="1.225707912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:26:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:26:43.470925258 +0000 UTC m=+433.722388730" watchObservedRunningTime="2026-10-30 22:26:43.471621286 +0000 UTC m=+433.723084757"
Oct 30 22:26:54.455605 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:26:54.455577    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-ml7lt"
Oct 30 22:27:07.515295 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:27:07.515247    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:27:37.514724 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:27:37.514688    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:28:07.514460 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:28:07.514422    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:28:37.515154 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:28:37.515046    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:29:07.515298 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:29:07.515250    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:29:37.515316 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:29:37.515275    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:30:07.515384 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:30:07.515342    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:30:37.514516 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:30:37.514475    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:31:07.517354 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:31:07.517289    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:31:37.514949 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:31:37.514847    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:32:07.515910 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:32:07.515816    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:32:37.515136 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:32:37.515091    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:33:07.516193 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:33:07.516152    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:33:36.359265 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.359229    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"]
Oct 30 22:33:36.362935 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.362884    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.376682 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.376660    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-default-dockercfg-kxprg\""
Oct 30 22:33:36.378791 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.378761    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 22:33:36.378933 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.378813    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 22:33:36.378933 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.378872    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 22:33:36.386132 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.386107    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"]
Oct 30 22:33:36.427338 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.427306    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.427338 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.427339    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.427549 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.427362    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.427549 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.427437    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc55ee32-394e-44f8-b286-ae33c1f29b23-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.427549 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.427524    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.427549 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.427545    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.427690 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.427587    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.427690 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.427606    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.427690 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.427648    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwk8j\" (UniqueName: \"kubernetes.io/projected/fc55ee32-394e-44f8-b286-ae33c1f29b23-kube-api-access-cwk8j\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528042 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528005    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528232 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528049    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528232 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528080    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528232 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528104    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc55ee32-394e-44f8-b286-ae33c1f29b23-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528232 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528143    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528232 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528169    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528232 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528209    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528530 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528290    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528530 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528321    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwk8j\" (UniqueName: \"kubernetes.io/projected/fc55ee32-394e-44f8-b286-ae33c1f29b23-kube-api-access-cwk8j\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528632 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528531    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528687 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528638    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528687 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528666    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.528771 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.528686    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.529126 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.529107    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc55ee32-394e-44f8-b286-ae33c1f29b23-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.530749 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.530723    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.530936 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.530920    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.542355 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.542329    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc55ee32-394e-44f8-b286-ae33c1f29b23-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.542460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.542436    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwk8j\" (UniqueName: \"kubernetes.io/projected/fc55ee32-394e-44f8-b286-ae33c1f29b23-kube-api-access-cwk8j\") pod \"openshift-ai-inference-openshift-default-86b669b864-njf8w\" (UID: \"fc55ee32-394e-44f8-b286-ae33c1f29b23\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.675456 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.675423    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:36.958838 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.958754    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"]
Oct 30 22:33:36.961844 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:33:36.961814    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc55ee32_394e_44f8_b286_ae33c1f29b23.slice/crio-0ad83d53d06b631595a3a05f7d7a82bd814513cf840f8060365ea179ade552b3 WatchSource:0}: Error finding container 0ad83d53d06b631595a3a05f7d7a82bd814513cf840f8060365ea179ade552b3: Status 404 returned error can't find the container with id 0ad83d53d06b631595a3a05f7d7a82bd814513cf840f8060365ea179ade552b3
Oct 30 22:33:36.963737 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:36.963721    5664 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:33:37.515280 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:33:37.515166    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:33:37.917758 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:37.917721    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" event={"ID":"fc55ee32-394e-44f8-b286-ae33c1f29b23","Type":"ContainerStarted","Data":"0ad83d53d06b631595a3a05f7d7a82bd814513cf840f8060365ea179ade552b3"}
Oct 30 22:33:40.150935 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:40.150875    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:33:40.151250 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:40.150975    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:33:40.151250 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:40.151003    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:33:40.932786 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:40.932746    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" event={"ID":"fc55ee32-394e-44f8-b286-ae33c1f29b23","Type":"ContainerStarted","Data":"385f091105d5ca9d0beb62f8a90f57ca8ee73a2b9e5158ec012c608d5c86634e"}
Oct 30 22:33:40.960526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:40.960472    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" podStartSLOduration=1.774290294 podStartE2EDuration="4.960458674s" totalImagesPullingTime="3.18616838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:40.958829237 +0000 UTC m=+851.210292707" watchObservedRunningTime="2026-10-30 22:33:40.960458674 +0000 UTC m=+851.211922212"
Oct 30 22:33:41.675849 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:41.675815    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:41.680600 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:41.680576    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:41.936559 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:41.936473    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:41.937465 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:41.937445    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:33:51.371528 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.371491    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"]
Oct 30 22:33:51.374583 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.374565    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.388249 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.388224    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"]
Oct 30 22:33:51.457256 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.457220    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.457256 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.457257    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf8b1942-8300-481c-9a58-e9afbeed93e5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.457490 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.457285    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.457490 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.457331    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk5rl\" (UniqueName: \"kubernetes.io/projected/cf8b1942-8300-481c-9a58-e9afbeed93e5-kube-api-access-dk5rl\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.457490 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.457453    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.457609 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.457496    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.457609 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.457514    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.457609 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.457531    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.457609 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.457547    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.558785 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.558742    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.558797    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.558821    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.558844    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.558866    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.558926    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.558954    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf8b1942-8300-481c-9a58-e9afbeed93e5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.558995    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.559019    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk5rl\" (UniqueName: \"kubernetes.io/projected/cf8b1942-8300-481c-9a58-e9afbeed93e5-kube-api-access-dk5rl\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.559249    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.559322    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.559351    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.559389    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.559727 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.559704    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf8b1942-8300-481c-9a58-e9afbeed93e5-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.561680 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.561657    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.561744 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.561723    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.572945 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.572922    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk5rl\" (UniqueName: \"kubernetes.io/projected/cf8b1942-8300-481c-9a58-e9afbeed93e5-kube-api-access-dk5rl\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.573057 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.572996    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf8b1942-8300-481c-9a58-e9afbeed93e5-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-nn2ft\" (UID: \"cf8b1942-8300-481c-9a58-e9afbeed93e5\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.687386 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.687338    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:51.965367 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.965294    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"]
Oct 30 22:33:51.968304 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:33:51.968277    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8b1942_8300_481c_9a58_e9afbeed93e5.slice/crio-b91ef07d30d926faae9fbb300c7828c168ae760c061a4cd6b3f90632949c3ac4 WatchSource:0}: Error finding container b91ef07d30d926faae9fbb300c7828c168ae760c061a4cd6b3f90632949c3ac4: Status 404 returned error can't find the container with id b91ef07d30d926faae9fbb300c7828c168ae760c061a4cd6b3f90632949c3ac4
Oct 30 22:33:51.971653 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.971621    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:33:51.971765 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.971704    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:33:51.971765 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.971752    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:33:51.977596 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:51.977570    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" event={"ID":"cf8b1942-8300-481c-9a58-e9afbeed93e5","Type":"ContainerStarted","Data":"b91ef07d30d926faae9fbb300c7828c168ae760c061a4cd6b3f90632949c3ac4"}
Oct 30 22:33:52.983393 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:52.983361    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" event={"ID":"cf8b1942-8300-481c-9a58-e9afbeed93e5","Type":"ContainerStarted","Data":"862a04a3b40de2d787a719b147c4bde8ad30c2635cb40a081178c200dcb65a5f"}
Oct 30 22:33:53.009968 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:53.009880    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" podStartSLOduration=2.009861908 podStartE2EDuration="2.009861908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:33:53.00876127 +0000 UTC m=+863.260224737" watchObservedRunningTime="2026-10-30 22:33:53.009861908 +0000 UTC m=+863.261325373"
Oct 30 22:33:53.688380 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:53.688333    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:53.693312 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:53.693288    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:53.987706 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:53.987616    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:33:53.988675 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:33:53.988655    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:34:07.516260 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:34:07.516223    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:34:35.159139 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.159105    5664 generic.go:356] "Generic (PLEG): container finished" podID="fc55ee32-394e-44f8-b286-ae33c1f29b23" containerID="385f091105d5ca9d0beb62f8a90f57ca8ee73a2b9e5158ec012c608d5c86634e" exitCode=0
Oct 30 22:34:35.159649 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.159189    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" event={"ID":"fc55ee32-394e-44f8-b286-ae33c1f29b23","Type":"ContainerDied","Data":"385f091105d5ca9d0beb62f8a90f57ca8ee73a2b9e5158ec012c608d5c86634e"}
Oct 30 22:34:35.159649 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.159567    5664 scope.go:122] "RemoveContainer" containerID="385f091105d5ca9d0beb62f8a90f57ca8ee73a2b9e5158ec012c608d5c86634e"
Oct 30 22:34:35.163455 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.163414    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:35.163594 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.163507    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:35.163594 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.163587    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:35.163733 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.163652    5664 generic.go:356] "Generic (PLEG): container finished" podID="cf8b1942-8300-481c-9a58-e9afbeed93e5" containerID="862a04a3b40de2d787a719b147c4bde8ad30c2635cb40a081178c200dcb65a5f" exitCode=0
Oct 30 22:34:35.163733 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.163680    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" event={"ID":"cf8b1942-8300-481c-9a58-e9afbeed93e5","Type":"ContainerDied","Data":"862a04a3b40de2d787a719b147c4bde8ad30c2635cb40a081178c200dcb65a5f"}
Oct 30 22:34:35.164031 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.164007    5664 scope.go:122] "RemoveContainer" containerID="862a04a3b40de2d787a719b147c4bde8ad30c2635cb40a081178c200dcb65a5f"
Oct 30 22:34:35.164965 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.164937    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:35.165051 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.165014    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:35.165103 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:35.165061    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:36.169770 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:36.169732    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" event={"ID":"fc55ee32-394e-44f8-b286-ae33c1f29b23","Type":"ContainerStarted","Data":"3e2bd0adb14f5038b158c5e191acb3d535c5006ff1a5c795e6b7c3649a1bd627"}
Oct 30 22:34:36.171621 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:36.171602    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" event={"ID":"cf8b1942-8300-481c-9a58-e9afbeed93e5","Type":"ContainerStarted","Data":"a900c2d358f911c821d147c59145e2ac5a8512780fdab43cedff72a9bf31fdf9"}
Oct 30 22:34:36.675997 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:36.675945    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:34:36.676201 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:36.676174    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-njf8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Oct 30 22:34:36.676253 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:36.676234    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" podUID="fc55ee32-394e-44f8-b286-ae33c1f29b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Oct 30 22:34:36.702924 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:36.699229    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:34:36.702924 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:36.700849    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-nn2ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 22:34:36.702924 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:36.700929    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" podUID="cf8b1942-8300-481c-9a58-e9afbeed93e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 22:34:37.546662 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:34:37.546627    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:34:37.676794 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:37.676754    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-njf8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Oct 30 22:34:37.676990 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:37.676806    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" podUID="fc55ee32-394e-44f8-b286-ae33c1f29b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Oct 30 22:34:37.690887 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:37.690851    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-nn2ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 22:34:37.691217 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:37.691185    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" podUID="cf8b1942-8300-481c-9a58-e9afbeed93e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 22:34:38.676118 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:38.676083    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-njf8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Oct 30 22:34:38.676513 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:38.676139    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" podUID="fc55ee32-394e-44f8-b286-ae33c1f29b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Oct 30 22:34:38.696925 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:38.695589    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-nn2ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 22:34:38.696925 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:38.695666    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" podUID="cf8b1942-8300-481c-9a58-e9afbeed93e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 22:34:38.991114 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:38.991008    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:34:39.676342 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:39.676235    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-njf8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Oct 30 22:34:39.676342 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:39.676317    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" podUID="fc55ee32-394e-44f8-b286-ae33c1f29b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Oct 30 22:34:39.694183 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:39.694143    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-nn2ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 22:34:39.694523 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:39.694477    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" podUID="cf8b1942-8300-481c-9a58-e9afbeed93e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 22:34:40.676837 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:40.676801    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-njf8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused" start-of-body=
Oct 30 22:34:40.677299 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:40.676864    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w" podUID="fc55ee32-394e-44f8-b286-ae33c1f29b23" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.21:15021/healthz/ready\": dial tcp 10.134.0.21:15021: connect: connection refused"
Oct 30 22:34:40.689864 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:40.689829    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-nn2ft container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused" start-of-body=
Oct 30 22:34:40.690177 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:40.690151    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft" podUID="cf8b1942-8300-481c-9a58-e9afbeed93e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.22:15021/healthz/ready\": dial tcp 10.134.0.22:15021: connect: connection refused"
Oct 30 22:34:41.680021 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:41.679987    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:34:41.680465 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:41.680301    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:34:41.681125 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:41.681106    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-njf8w"
Oct 30 22:34:41.692010 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:41.691985    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:34:41.692804 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:41.692791    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-nn2ft"
Oct 30 22:34:51.502032 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.501998    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"]
Oct 30 22:34:51.505354 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.505331    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.519888 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.519861    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"]
Oct 30 22:34:51.551818 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.551786    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.552062 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.551837    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.552062 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.551949    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.552062 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.552009    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.552062 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.552083    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da38088-f4e1-4c68-ba94-b6a94434a021-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.552443 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.552160    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m698r\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-kube-api-access-m698r\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.552443 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.552257    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.552443 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.552304    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.552443 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.552338    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.652967 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.652925    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653170 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.652979    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653170 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.652999    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653170 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653027    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da38088-f4e1-4c68-ba94-b6a94434a021-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653170 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653045    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m698r\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-kube-api-access-m698r\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653170 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653078    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653419 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653233    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653419 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653293    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653419 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653323    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653455    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653490    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653570 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653533    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653769 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653747    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.653828 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.653799    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da38088-f4e1-4c68-ba94-b6a94434a021-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.655748 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.655731    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.655824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.655732    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.666630 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.666602    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.666810 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.666793    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m698r\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-kube-api-access-m698r\") pod \"openshift-ai-inference-openshift-default-86b669b864-kfmds\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:51.819100 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:51.819010    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:52.108075 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:52.107993    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"]
Oct 30 22:34:52.111114 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:34:52.111082    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da38088_f4e1_4c68_ba94_b6a94434a021.slice/crio-a28d2baf569c74ea822aa6ef0b4547442dad443cfb322537cb8435ce93444f5d WatchSource:0}: Error finding container a28d2baf569c74ea822aa6ef0b4547442dad443cfb322537cb8435ce93444f5d: Status 404 returned error can't find the container with id a28d2baf569c74ea822aa6ef0b4547442dad443cfb322537cb8435ce93444f5d
Oct 30 22:34:52.113247 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:52.113215    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:52.113313 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:52.113284    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:52.113349 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:52.113312    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:34:52.275316 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:52.275277    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" event={"ID":"7da38088-f4e1-4c68-ba94-b6a94434a021","Type":"ContainerStarted","Data":"d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2"}
Oct 30 22:34:52.275495 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:52.275335    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" event={"ID":"7da38088-f4e1-4c68-ba94-b6a94434a021","Type":"ContainerStarted","Data":"a28d2baf569c74ea822aa6ef0b4547442dad443cfb322537cb8435ce93444f5d"}
Oct 30 22:34:52.303733 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:52.303682    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" podStartSLOduration=1.303668786 podStartE2EDuration="1.303668786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:34:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:34:52.302133867 +0000 UTC m=+922.553597372" watchObservedRunningTime="2026-10-30 22:34:52.303668786 +0000 UTC m=+922.555132254"
Oct 30 22:34:52.819207 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:52.819163    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:53.820767 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:53.820732    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-kfmds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Oct 30 22:34:53.821179 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:53.820785    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Oct 30 22:34:54.819641 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:54.819602    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-kfmds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Oct 30 22:34:54.819817 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:54.819661    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Oct 30 22:34:55.819637 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:55.819595    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-kfmds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Oct 30 22:34:55.820083 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:55.819653    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Oct 30 22:34:56.820142 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:56.820111    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-kfmds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Oct 30 22:34:56.820534 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:56.820160    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Oct 30 22:34:57.819448 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:57.819415    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-kfmds container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused" start-of-body=
Oct 30 22:34:57.819600 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:57.819472    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.23:15021/healthz/ready\": dial tcp 10.134.0.23:15021: connect: connection refused"
Oct 30 22:34:58.823985 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:58.823954    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:58.824380 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:58.824247    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:34:58.824927 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:34:58.824887    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 22:35:07.516366 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:35:07.516328    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:35:37.515017 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:35:37.514976    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:36:07.515579 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:36:07.515538    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:36:37.516225 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:36:37.516188    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:37:07.515779 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:37:07.515734    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:37:37.569796 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:37:37.569760    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:38:07.515533 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:38:07.515458    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:38:37.515267 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:38:37.515224    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:39:07.515167 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:39:07.515131    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:39:33.504690 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.504659    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"]
Oct 30 22:39:33.507926 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.507890    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.514851 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.514813    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"scheduler-ha-replicas-test-kserve-self-signed-certs\""
Oct 30 22:39:33.514851 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.514813    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"default-dockercfg-5lf77\""
Oct 30 22:39:33.515034 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.514865    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"openshift-service-ca.crt\""
Oct 30 22:39:33.515034 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.514881    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-2541cd65\"/\"kube-root-ca.crt\""
Oct 30 22:39:33.518885 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.518849    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"]
Oct 30 22:39:33.625510 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.625475    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.625695 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.625525    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-dshm\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.625695 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.625548    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-home\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.625773 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.625683    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-model-cache\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.625773 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.625726    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d56d0db-6721-4947-ad65-a6e995339233-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.625773 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.625754    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzgp4\" (UniqueName: \"kubernetes.io/projected/0d56d0db-6721-4947-ad65-a6e995339233-kube-api-access-hzgp4\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.726608 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.726557    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-dshm\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.726608 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.726610    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-home\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.726860 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.726665    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-model-cache\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.726860 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.726702    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d56d0db-6721-4947-ad65-a6e995339233-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.726860 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.726733    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzgp4\" (UniqueName: \"kubernetes.io/projected/0d56d0db-6721-4947-ad65-a6e995339233-kube-api-access-hzgp4\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.726860 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.726770    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.727092 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.727071    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-home\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.727152 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.727116    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-model-cache\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.727230 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.727210    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-tmp-dir\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.729350 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.729328    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-dshm\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.729633 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.729614    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d56d0db-6721-4947-ad65-a6e995339233-tls-certs\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.744827 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.744795    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzgp4\" (UniqueName: \"kubernetes.io/projected/0d56d0db-6721-4947-ad65-a6e995339233-kube-api-access-hzgp4\") pod \"scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") " pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:33.820642 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:33.820551    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:34.096067 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:34.095981    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"]
Oct 30 22:39:34.099047 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:39:34.099017    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d56d0db_6721_4947_ad65_a6e995339233.slice/crio-176a56bfd27bab58f4d23558e1c793930bb65768e457d7f1d8ad5c370fba007c WatchSource:0}: Error finding container 176a56bfd27bab58f4d23558e1c793930bb65768e457d7f1d8ad5c370fba007c: Status 404 returned error can't find the container with id 176a56bfd27bab58f4d23558e1c793930bb65768e457d7f1d8ad5c370fba007c
Oct 30 22:39:34.100983 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:34.100968    5664 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:39:34.461396 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:34.461362    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz" event={"ID":"0d56d0db-6721-4947-ad65-a6e995339233","Type":"ContainerStarted","Data":"176a56bfd27bab58f4d23558e1c793930bb65768e457d7f1d8ad5c370fba007c"}
Oct 30 22:39:36.477019 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:36.476982    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz" event={"ID":"0d56d0db-6721-4947-ad65-a6e995339233","Type":"ContainerStarted","Data":"871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9"}
Oct 30 22:39:36.507303 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:36.507254    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz" podStartSLOduration=1.489424475 podStartE2EDuration="3.507238868s" totalImagesPullingTime="2.017814393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:39:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:39:36.505506215 +0000 UTC m=+1206.756969683" watchObservedRunningTime="2026-10-30 22:39:36.507238868 +0000 UTC m=+1206.758702336"
Oct 30 22:39:36.820817 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:36.820732    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:37.515174 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:39:37.515136    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:39:43.820973 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:43.820933    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:43.834215 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:43.834181    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:39:44.520393 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:39:44.520365    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:40:07.520135 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:40:07.520093    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:40:07.825833 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:07.825746    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"]
Oct 30 22:40:07.837364 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:40:07.837324    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9" cmd=["/bin/sleep","15"]
Oct 30 22:40:07.837544 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:07.837395    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:40:07.837544 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:40:07.837413    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz" podUID="0d56d0db-6721-4947-ad65-a6e995339233" containerName="main" containerID="cri-o://871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9"
Oct 30 22:40:07.837544 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:07.837460    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz" podUID="0d56d0db-6721-4947-ad65-a6e995339233" containerName="main" containerID="cri-o://871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9" gracePeriod=60
Oct 30 22:40:08.191409 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.191378    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:40:08.329250 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.329219    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-tmp-dir\") pod \"0d56d0db-6721-4947-ad65-a6e995339233\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") "
Oct 30 22:40:08.329452 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.329281    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-dshm\") pod \"0d56d0db-6721-4947-ad65-a6e995339233\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") "
Oct 30 22:40:08.329452 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.329302    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d56d0db-6721-4947-ad65-a6e995339233-kube-api-access-hzgp4\" (UniqueName: \"kubernetes.io/projected/0d56d0db-6721-4947-ad65-a6e995339233-kube-api-access-hzgp4\") pod \"0d56d0db-6721-4947-ad65-a6e995339233\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") "
Oct 30 22:40:08.329452 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.329334    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d56d0db-6721-4947-ad65-a6e995339233-tls-certs\" (UniqueName: \"kubernetes.io/secret/0d56d0db-6721-4947-ad65-a6e995339233-tls-certs\") pod \"0d56d0db-6721-4947-ad65-a6e995339233\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") "
Oct 30 22:40:08.329452 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.329362    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-model-cache\") pod \"0d56d0db-6721-4947-ad65-a6e995339233\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") "
Oct 30 22:40:08.329452 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.329378    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-home\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-home\") pod \"0d56d0db-6721-4947-ad65-a6e995339233\" (UID: \"0d56d0db-6721-4947-ad65-a6e995339233\") "
Oct 30 22:40:08.329716 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.329607    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-tmp-dir" pod "0d56d0db-6721-4947-ad65-a6e995339233" (UID: "0d56d0db-6721-4947-ad65-a6e995339233"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:08.329716 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.329642    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-model-cache" pod "0d56d0db-6721-4947-ad65-a6e995339233" (UID: "0d56d0db-6721-4947-ad65-a6e995339233"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:08.329830 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.329808    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-home" pod "0d56d0db-6721-4947-ad65-a6e995339233" (UID: "0d56d0db-6721-4947-ad65-a6e995339233"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:08.331729 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.331705    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-dshm" pod "0d56d0db-6721-4947-ad65-a6e995339233" (UID: "0d56d0db-6721-4947-ad65-a6e995339233"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:08.331843 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.331826    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d56d0db-6721-4947-ad65-a6e995339233-kube-api-access-hzgp4" pod "0d56d0db-6721-4947-ad65-a6e995339233" (UID: "0d56d0db-6721-4947-ad65-a6e995339233"). InnerVolumeSpecName "kube-api-access-hzgp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:08.331843 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.331832    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d56d0db-6721-4947-ad65-a6e995339233-tls-certs" pod "0d56d0db-6721-4947-ad65-a6e995339233" (UID: "0d56d0db-6721-4947-ad65-a6e995339233"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:08.430877 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.430843    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:08.430877 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.430874    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:08.431115 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.430888    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzgp4\" (UniqueName: \"kubernetes.io/projected/0d56d0db-6721-4947-ad65-a6e995339233-kube-api-access-hzgp4\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:08.431115 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.430929    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d56d0db-6721-4947-ad65-a6e995339233-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:08.431115 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.430942    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:08.431115 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.430954    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0d56d0db-6721-4947-ad65-a6e995339233-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:08.626639 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.626604    5664 generic.go:356] "Generic (PLEG): container finished" podID="0d56d0db-6721-4947-ad65-a6e995339233" containerID="871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9" exitCode=0
Oct 30 22:40:08.626639 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.626640    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz" event={"ID":"0d56d0db-6721-4947-ad65-a6e995339233","Type":"ContainerDied","Data":"871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9"}
Oct 30 22:40:08.627176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.626670    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"
Oct 30 22:40:08.627176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.626705    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz" event={"ID":"0d56d0db-6721-4947-ad65-a6e995339233","Type":"ContainerDied","Data":"176a56bfd27bab58f4d23558e1c793930bb65768e457d7f1d8ad5c370fba007c"}
Oct 30 22:40:08.627176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.626729    5664 scope.go:122] "RemoveContainer" containerID="871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9"
Oct 30 22:40:08.635778 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.635761    5664 scope.go:122] "RemoveContainer" containerID="871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9"
Oct 30 22:40:08.636066 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:40:08.636049    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9\": container with ID starting with 871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9 not found: ID does not exist" containerID="871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9"
Oct 30 22:40:08.636130 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.636074    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9"} err="failed to get container status \"871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9\": rpc error: code = NotFound desc = could not find container \"871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9\": container with ID starting with 871c786abc38ea783d09b8765ad430ef759052cf0c4ff1ee7084aab7a126bbd9 not found: ID does not exist"
Oct 30 22:40:08.648388 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.648366    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"]
Oct 30 22:40:08.655020 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:08.654997    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-2541cd65/scheduler-ha-replicas-test-kserve-74cd6f7448-tqfnz"]
Oct 30 22:40:09.867373 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:09.867342    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d56d0db-6721-4947-ad65-a6e995339233" path="/var/lib/kubelet/pods/0d56d0db-6721-4947-ad65-a6e995339233/volumes"
Oct 30 22:40:37.515145 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:40:37.515102    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:40:39.105019 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.104872    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"]
Oct 30 22:40:39.105407 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.105208    5664 cpu_manager.go:414] "containerMap: removing container" podUID="0d56d0db-6721-4947-ad65-a6e995339233" containerName="main"
Oct 30 22:40:39.105407 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.105221    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d56d0db-6721-4947-ad65-a6e995339233" containerName="main"
Oct 30 22:40:39.105407 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.105306    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d56d0db-6721-4947-ad65-a6e995339233" containerName="main"
Oct 30 22:40:39.109596 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.109575    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.115419 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.115396    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"kube-root-ca.crt\""
Oct 30 22:40:39.117460 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.117439    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-gw-ca-root-cert\""
Oct 30 22:40:39.118860 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.118835    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"router-gateway-1-openshift-default-dockercfg-hhjcj\""
Oct 30 22:40:39.129540 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.129512    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"]
Oct 30 22:40:39.194719 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.194683    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-envoy\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.194946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.194761    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-data\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.194946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.194792    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e2bf924-1875-426a-a602-b164a4e74b57-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.194946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.194891    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp6f8\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-kube-api-access-cp6f8\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.194946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.194946    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-credential-socket\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.195161 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.194967    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-certs\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.195161 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.195061    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-socket\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.195161 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.195131    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e2bf924-1875-426a-a602-b164a4e74b57-istio-podinfo\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.195161 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.195151    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-istio-token\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.214815 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.214788    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"openshift-service-ca.crt\""
Oct 30 22:40:39.296133 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296096    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp6f8\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-kube-api-access-cp6f8\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296141    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-credential-socket\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296183    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-certs\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296216    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-socket\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296260    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e2bf924-1875-426a-a602-b164a4e74b57-istio-podinfo\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296284    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-istio-token\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296592 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296335    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-envoy\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296592 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296356    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-data\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296592 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296379    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e2bf924-1875-426a-a602-b164a4e74b57-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296771 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296647    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-socket\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296771 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296696    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-credential-socket\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.296979 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.296959    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-certs\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.297067 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.297008    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-data\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.297219 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.297201    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e2bf924-1875-426a-a602-b164a4e74b57-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.298802 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.298781    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-envoy\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.298914 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.298827    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e2bf924-1875-426a-a602-b164a4e74b57-istio-podinfo\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.314759 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.314735    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp6f8\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-kube-api-access-cp6f8\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.317907 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.317870    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-istio-token\") pod \"router-gateway-1-openshift-default-96f68d6bf-4kbqg\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.423438 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.423397    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:39.706853 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.706727    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"]
Oct 30 22:40:39.711619 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:40:39.711585    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e2bf924_1875_426a_a602_b164a4e74b57.slice/crio-5ce11eeef272e0f53ec8cf2840c1d79a6f3d34898e9116e13007a1348fbd4e96 WatchSource:0}: Error finding container 5ce11eeef272e0f53ec8cf2840c1d79a6f3d34898e9116e13007a1348fbd4e96: Status 404 returned error can't find the container with id 5ce11eeef272e0f53ec8cf2840c1d79a6f3d34898e9116e13007a1348fbd4e96
Oct 30 22:40:39.713620 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.713587    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:40:39.713719 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.713685    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:40:39.713785 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.713747    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 22:40:39.749758 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:39.749724    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg" event={"ID":"6e2bf924-1875-426a-a602-b164a4e74b57","Type":"ContainerStarted","Data":"5ce11eeef272e0f53ec8cf2840c1d79a6f3d34898e9116e13007a1348fbd4e96"}
Oct 30 22:40:40.755612 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:40.755577    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg" event={"ID":"6e2bf924-1875-426a-a602-b164a4e74b57","Type":"ContainerStarted","Data":"21703536ea9872ca66a03dfaacba17794a8fdd163e562e6ae4e52062d77e25ed"}
Oct 30 22:40:40.785508 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:40.785456    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg" podStartSLOduration=1.785442272 podStartE2EDuration="1.785442272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:40:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:40.78344675 +0000 UTC m=+1271.034910231" watchObservedRunningTime="2026-10-30 22:40:40.785442272 +0000 UTC m=+1271.036905740"
Oct 30 22:40:41.424202 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:41.424163    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:41.429058 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:41.429033    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:41.759829 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:41.759747    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:41.760929 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:41.760888    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:46.590584 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.590542    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"]
Oct 30 22:40:46.600313 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.600286    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.605953 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.605928    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs\""
Oct 30 22:40:46.606104 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.605966    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-gateway-section-name-propagation-49b5edd9\"/\"default-dockercfg-h6rkl\""
Oct 30 22:40:46.607491 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.607423    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"]
Oct 30 22:40:46.663264 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.663228    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.663442 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.663273    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.663442 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.663348    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.663442 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.663408    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6kgk\" (UniqueName: \"kubernetes.io/projected/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-kube-api-access-x6kgk\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.663564 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.663455    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.663564 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.663471    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.763941 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.763878    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.763941 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.763941    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.764178 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.763992    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.764178 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.764011    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.764178 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.764034    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.764178 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.764085    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6kgk\" (UniqueName: \"kubernetes.io/projected/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-kube-api-access-x6kgk\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.764412 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.764388    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-model-cache\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.764478 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.764433    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tmp-dir\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.764478 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.764457    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-home\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.766572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.766539    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-dshm\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.766765 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.766746    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tls-certs\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.777955 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.777888    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6kgk\" (UniqueName: \"kubernetes.io/projected/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-kube-api-access-x6kgk\") pod \"gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") " pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:46.917548 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:46.917505    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:47.193957 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:47.193843    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"]
Oct 30 22:40:47.196769 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:40:47.196737    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaa7a3ff_a04d_434e_8c93_1a3d21f573cb.slice/crio-65e13d0921dd52b76b8ba82f349038913fbd4ab398a369eae6177e3af43d0065 WatchSource:0}: Error finding container 65e13d0921dd52b76b8ba82f349038913fbd4ab398a369eae6177e3af43d0065: Status 404 returned error can't find the container with id 65e13d0921dd52b76b8ba82f349038913fbd4ab398a369eae6177e3af43d0065
Oct 30 22:40:47.788993 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:47.788948    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz" event={"ID":"daa7a3ff-a04d-434e-8c93-1a3d21f573cb","Type":"ContainerStarted","Data":"a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020"}
Oct 30 22:40:47.789401 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:47.789029    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz" event={"ID":"daa7a3ff-a04d-434e-8c93-1a3d21f573cb","Type":"ContainerStarted","Data":"65e13d0921dd52b76b8ba82f349038913fbd4ab398a369eae6177e3af43d0065"}
Oct 30 22:40:47.917641 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:47.917610    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:52.993813 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:52.992615    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz" podStartSLOduration=6.99259816 podStartE2EDuration="6.99259816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:40:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:40:47.810396034 +0000 UTC m=+1278.061859504" watchObservedRunningTime="2026-10-30 22:40:52.99259816 +0000 UTC m=+1283.244061629"
Oct 30 22:40:52.993813 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:52.993745    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"]
Oct 30 22:40:53.005048 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.005012    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"]
Oct 30 22:40:53.005283 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.005243    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg" podUID="6e2bf924-1875-426a-a602-b164a4e74b57" containerName="istio-proxy" containerID="cri-o://21703536ea9872ca66a03dfaacba17794a8fdd163e562e6ae4e52062d77e25ed" gracePeriod=30
Oct 30 22:40:53.013765 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:40:53.013730    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020" cmd=["/bin/sleep","15"]
Oct 30 22:40:53.013865 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.013783    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:40:53.013865 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:40:53.013797    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz" podUID="daa7a3ff-a04d-434e-8c93-1a3d21f573cb" containerName="main" containerID="cri-o://a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020"
Oct 30 22:40:53.013865 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.013844    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz" podUID="daa7a3ff-a04d-434e-8c93-1a3d21f573cb" containerName="main" containerID="cri-o://a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020" gracePeriod=60
Oct 30 22:40:53.325107 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.325080    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:53.446488 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.446458    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-model-cache\") pod \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") "
Oct 30 22:40:53.446694 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.446501    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tls-certs\" (UniqueName: \"kubernetes.io/secret/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tls-certs\") pod \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") "
Oct 30 22:40:53.446694 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.446567    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-dshm\") pod \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") "
Oct 30 22:40:53.446694 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.446600    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-home\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-home\") pod \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") "
Oct 30 22:40:53.446694 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.446633    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-kube-api-access-x6kgk\" (UniqueName: \"kubernetes.io/projected/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-kube-api-access-x6kgk\") pod \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") "
Oct 30 22:40:53.446694 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.446659    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tmp-dir\") pod \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\" (UID: \"daa7a3ff-a04d-434e-8c93-1a3d21f573cb\") "
Oct 30 22:40:53.446966 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.446780    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-model-cache" pod "daa7a3ff-a04d-434e-8c93-1a3d21f573cb" (UID: "daa7a3ff-a04d-434e-8c93-1a3d21f573cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:53.447022 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.446986    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:53.447070 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.447046    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tmp-dir" pod "daa7a3ff-a04d-434e-8c93-1a3d21f573cb" (UID: "daa7a3ff-a04d-434e-8c93-1a3d21f573cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:53.447105 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.447058    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-home" pod "daa7a3ff-a04d-434e-8c93-1a3d21f573cb" (UID: "daa7a3ff-a04d-434e-8c93-1a3d21f573cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:53.449004 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.448976    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-kube-api-access-x6kgk" pod "daa7a3ff-a04d-434e-8c93-1a3d21f573cb" (UID: "daa7a3ff-a04d-434e-8c93-1a3d21f573cb"). InnerVolumeSpecName "kube-api-access-x6kgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:53.449189 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.449168    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-dshm" pod "daa7a3ff-a04d-434e-8c93-1a3d21f573cb" (UID: "daa7a3ff-a04d-434e-8c93-1a3d21f573cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:53.449254 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.449244    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tls-certs" pod "daa7a3ff-a04d-434e-8c93-1a3d21f573cb" (UID: "daa7a3ff-a04d-434e-8c93-1a3d21f573cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:40:53.548110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.548024    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:53.548110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.548052    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:53.548110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.548062    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6kgk\" (UniqueName: \"kubernetes.io/projected/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-kube-api-access-x6kgk\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:53.548110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.548074    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:53.548110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.548083    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/daa7a3ff-a04d-434e-8c93-1a3d21f573cb-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:53.830008 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.829924    5664 generic.go:356] "Generic (PLEG): container finished" podID="daa7a3ff-a04d-434e-8c93-1a3d21f573cb" containerID="a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020" exitCode=0
Oct 30 22:40:53.830008 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.829968    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"
Oct 30 22:40:53.830189 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.829968    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz" event={"ID":"daa7a3ff-a04d-434e-8c93-1a3d21f573cb","Type":"ContainerDied","Data":"a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020"}
Oct 30 22:40:53.830189 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.830095    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz" event={"ID":"daa7a3ff-a04d-434e-8c93-1a3d21f573cb","Type":"ContainerDied","Data":"65e13d0921dd52b76b8ba82f349038913fbd4ab398a369eae6177e3af43d0065"}
Oct 30 22:40:53.830189 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.830120    5664 scope.go:122] "RemoveContainer" containerID="a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020"
Oct 30 22:40:53.839399 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.839381    5664 scope.go:122] "RemoveContainer" containerID="a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020"
Oct 30 22:40:53.839680 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:40:53.839662    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020\": container with ID starting with a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020 not found: ID does not exist" containerID="a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020"
Oct 30 22:40:53.839745 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.839689    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020"} err="failed to get container status \"a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020\": rpc error: code = NotFound desc = could not find container \"a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020\": container with ID starting with a061a5a93759c25e787daeea0e41dcfb7dec817c090d8466e9572c109c5b0020 not found: ID does not exist"
Oct 30 22:40:53.853449 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.853428    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"]
Oct 30 22:40:53.860102 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.860077    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/gw-section-name-router-with-gat-2f0a622e-kserve-87cb66947-gnpgz"]
Oct 30 22:40:53.867527 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:53.867505    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daa7a3ff-a04d-434e-8c93-1a3d21f573cb" path="/var/lib/kubelet/pods/daa7a3ff-a04d-434e-8c93-1a3d21f573cb/volumes"
Oct 30 22:40:56.761887 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:56.761847    5664 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg" podUID="6e2bf924-1875-426a-a602-b164a4e74b57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Oct 30 22:40:56.844769 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:56.844728    5664 generic.go:356] "Generic (PLEG): container finished" podID="6e2bf924-1875-426a-a602-b164a4e74b57" containerID="21703536ea9872ca66a03dfaacba17794a8fdd163e562e6ae4e52062d77e25ed" exitCode=0
Oct 30 22:40:56.844932 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:56.844800    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg" event={"ID":"6e2bf924-1875-426a-a602-b164a4e74b57","Type":"ContainerDied","Data":"21703536ea9872ca66a03dfaacba17794a8fdd163e562e6ae4e52062d77e25ed"}
Oct 30 22:40:56.947533 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:56.947500    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:57.081355 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081316    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-envoy\") pod \"6e2bf924-1875-426a-a602-b164a4e74b57\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") "
Oct 30 22:40:57.081535 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081384    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-data\") pod \"6e2bf924-1875-426a-a602-b164a4e74b57\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") "
Oct 30 22:40:57.081535 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081409    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-socket\") pod \"6e2bf924-1875-426a-a602-b164a4e74b57\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") "
Oct 30 22:40:57.081535 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081440    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-kube-api-access-cp6f8\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-kube-api-access-cp6f8\") pod \"6e2bf924-1875-426a-a602-b164a4e74b57\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") "
Oct 30 22:40:57.081535 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081466    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-istio-token\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-istio-token\") pod \"6e2bf924-1875-426a-a602-b164a4e74b57\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") "
Oct 30 22:40:57.081535 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081492    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e2bf924-1875-426a-a602-b164a4e74b57-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e2bf924-1875-426a-a602-b164a4e74b57-istiod-ca-cert\") pod \"6e2bf924-1875-426a-a602-b164a4e74b57\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") "
Oct 30 22:40:57.081535 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081525    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6e2bf924-1875-426a-a602-b164a4e74b57-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e2bf924-1875-426a-a602-b164a4e74b57-istio-podinfo\") pod \"6e2bf924-1875-426a-a602-b164a4e74b57\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") "
Oct 30 22:40:57.081840 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081653    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-data" pod "6e2bf924-1875-426a-a602-b164a4e74b57" (UID: "6e2bf924-1875-426a-a602-b164a4e74b57"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:57.081840 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081659    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-credential-socket\") pod \"6e2bf924-1875-426a-a602-b164a4e74b57\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") "
Oct 30 22:40:57.081840 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081808    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-certs\") pod \"6e2bf924-1875-426a-a602-b164a4e74b57\" (UID: \"6e2bf924-1875-426a-a602-b164a4e74b57\") "
Oct 30 22:40:57.082044 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081821    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-socket" pod "6e2bf924-1875-426a-a602-b164a4e74b57" (UID: "6e2bf924-1875-426a-a602-b164a4e74b57"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:57.082044 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.081930    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-credential-socket" pod "6e2bf924-1875-426a-a602-b164a4e74b57" (UID: "6e2bf924-1875-426a-a602-b164a4e74b57"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:57.082135 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.082109    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-certs" pod "6e2bf924-1875-426a-a602-b164a4e74b57" (UID: "6e2bf924-1875-426a-a602-b164a4e74b57"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:57.082190 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.082167    5664 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-data\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.082240 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.082188    5664 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-socket\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.082240 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.082204    5664 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-credential-socket\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.082398 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.082369    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e2bf924-1875-426a-a602-b164a4e74b57-istiod-ca-cert" pod "6e2bf924-1875-426a-a602-b164a4e74b57" (UID: "6e2bf924-1875-426a-a602-b164a4e74b57"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:40:57.083720 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.083697    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-envoy" pod "6e2bf924-1875-426a-a602-b164a4e74b57" (UID: "6e2bf924-1875-426a-a602-b164a4e74b57"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:40:57.083802 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.083749    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-istio-token" pod "6e2bf924-1875-426a-a602-b164a4e74b57" (UID: "6e2bf924-1875-426a-a602-b164a4e74b57"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:57.083872 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.083854    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-kube-api-access-cp6f8" pod "6e2bf924-1875-426a-a602-b164a4e74b57" (UID: "6e2bf924-1875-426a-a602-b164a4e74b57"). InnerVolumeSpecName "kube-api-access-cp6f8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:40:57.084237 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.084223    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e2bf924-1875-426a-a602-b164a4e74b57-istio-podinfo" pod "6e2bf924-1875-426a-a602-b164a4e74b57" (UID: "6e2bf924-1875-426a-a602-b164a4e74b57"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 22:40:57.182738 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.182701    5664 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-workload-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.182738 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.182731    5664 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e2bf924-1875-426a-a602-b164a4e74b57-istio-envoy\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.182738 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.182740    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cp6f8\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-kube-api-access-cp6f8\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.183010 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.182751    5664 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e2bf924-1875-426a-a602-b164a4e74b57-istio-token\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.183010 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.182760    5664 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e2bf924-1875-426a-a602-b164a4e74b57-istiod-ca-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.183010 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.182768    5664 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e2bf924-1875-426a-a602-b164a4e74b57-istio-podinfo\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:40:57.849837 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.849798    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg" event={"ID":"6e2bf924-1875-426a-a602-b164a4e74b57","Type":"ContainerDied","Data":"5ce11eeef272e0f53ec8cf2840c1d79a6f3d34898e9116e13007a1348fbd4e96"}
Oct 30 22:40:57.850415 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.849865    5664 scope.go:122] "RemoveContainer" containerID="21703536ea9872ca66a03dfaacba17794a8fdd163e562e6ae4e52062d77e25ed"
Oct 30 22:40:57.850415 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.849809    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"
Oct 30 22:40:57.872880 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.872854    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"]
Oct 30 22:40:57.877822 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:57.877796    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-gateway-section-name-propagation-49b5edd9/router-gateway-1-openshift-default-96f68d6bf-4kbqg"]
Oct 30 22:40:59.868037 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:40:59.867999    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e2bf924-1875-426a-a602-b164a4e74b57" path="/var/lib/kubelet/pods/6e2bf924-1875-426a-a602-b164a4e74b57/volumes"
Oct 30 22:41:07.516207 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:41:07.516168    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:41:37.515049 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:41:37.515007    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:41:46.262811 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:41:46.262763    5664 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" podUID="320724d28f4bb4eb7790f5cce736ad83" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 22:42:07.515178 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:42:07.515143    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:42:32.134801 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.134711    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"]
Oct 30 22:42:32.135260 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.135067    5664 cpu_manager.go:414] "containerMap: removing container" podUID="daa7a3ff-a04d-434e-8c93-1a3d21f573cb" containerName="main"
Oct 30 22:42:32.135260 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.135080    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daa7a3ff-a04d-434e-8c93-1a3d21f573cb" containerName="main"
Oct 30 22:42:32.135260 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.135100    5664 cpu_manager.go:414] "containerMap: removing container" podUID="6e2bf924-1875-426a-a602-b164a4e74b57" containerName="istio-proxy"
Oct 30 22:42:32.135260 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.135104    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e2bf924-1875-426a-a602-b164a4e74b57" containerName="istio-proxy"
Oct 30 22:42:32.135260 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.135154    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="daa7a3ff-a04d-434e-8c93-1a3d21f573cb" containerName="main"
Oct 30 22:42:32.135260 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.135165    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e2bf924-1875-426a-a602-b164a4e74b57" containerName="istio-proxy"
Oct 30 22:42:32.138081 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.138065    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.150210 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.150187    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"openshift-service-ca.crt\""
Oct 30 22:42:32.150324 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.150296    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"kube-root-ca.crt\""
Oct 30 22:42:32.159388 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.159361    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"]
Oct 30 22:42:32.161594 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.161572    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-home\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.161720 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.161600    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.161720 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.161627    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.161720 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.161646    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2s6gh\" (UniqueName: \"kubernetes.io/projected/b577cf28-c807-4d34-9f7e-af47e6f81c48-kube-api-access-2s6gh\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.161846 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.161795    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.161846 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.161825    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b577cf28-c807-4d34-9f7e-af47e6f81c48-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.241819 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.241788    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-dockercfg-76j4f\""
Oct 30 22:42:32.241819 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.241792    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"default-dockercfg-vpmlk\""
Oct 30 22:42:32.241819 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.241795    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Oct 30 22:42:32.262246 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.262213    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-home\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.262246 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.262247    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.262422 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.262270    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.262422 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.262298    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2s6gh\" (UniqueName: \"kubernetes.io/projected/b577cf28-c807-4d34-9f7e-af47e6f81c48-kube-api-access-2s6gh\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.262422 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.262365    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.262422 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.262387    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b577cf28-c807-4d34-9f7e-af47e6f81c48-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.262699 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.262677    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-home\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.262784 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.262759    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.262922 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.262883    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.264636 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.264616    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.264918 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.264885    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b577cf28-c807-4d34-9f7e-af47e6f81c48-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.275986 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.275963    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2s6gh\" (UniqueName: \"kubernetes.io/projected/b577cf28-c807-4d34-9f7e-af47e6f81c48-kube-api-access-2s6gh\") pod \"scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.452449 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.452404    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:32.745289 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:32.745261    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"]
Oct 30 22:42:32.747162 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:42:32.747132    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb577cf28_c807_4d34_9f7e_af47e6f81c48.slice/crio-853b46acf4eb578d306f1106025cdf434fefa03ff1551cda3983c4da2f5746bd WatchSource:0}: Error finding container 853b46acf4eb578d306f1106025cdf434fefa03ff1551cda3983c4da2f5746bd: Status 404 returned error can't find the container with id 853b46acf4eb578d306f1106025cdf434fefa03ff1551cda3983c4da2f5746bd
Oct 30 22:42:33.354127 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:33.354073    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" event={"ID":"b577cf28-c807-4d34-9f7e-af47e6f81c48","Type":"ContainerStarted","Data":"853b46acf4eb578d306f1106025cdf434fefa03ff1551cda3983c4da2f5746bd"}
Oct 30 22:42:37.373837 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:37.373273    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" event={"ID":"b577cf28-c807-4d34-9f7e-af47e6f81c48","Type":"ContainerStarted","Data":"bfebe705d0eddf17a615c41df8fb538dcd3cd2fe10dd41f2d71090416337b5b8"}
Oct 30 22:42:37.373837 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:37.373500    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:37.574343 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:42:37.574309    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:42:38.383126 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:38.382193    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" event={"ID":"b577cf28-c807-4d34-9f7e-af47e6f81c48","Type":"ContainerStarted","Data":"8f36e2971a71dab9b08750f9c53ab11d7b4b8ea6063cf157377ab0e6dc7f4cf1"}
Oct 30 22:42:38.453120 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:38.453031    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:42.453602 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:42.453565    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:42.465983 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:42.465957    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:42.491625 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:42.491576    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" podStartSLOduration=6.775009588 podStartE2EDuration="10.491563642s" totalImagesPullingTime="3.716554054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:42:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:42:38.41116738 +0000 UTC m=+1388.662630849" watchObservedRunningTime="2026-10-30 22:42:42.491563642 +0000 UTC m=+1392.743027109"
Oct 30 22:42:43.416109 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:43.416080    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:42:54.420240 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:42:54.420201    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:43:07.516229 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:43:07.516192    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:43:13.072205 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.072165    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"]
Oct 30 22:43:13.082013 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:43:13.081974    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="8f36e2971a71dab9b08750f9c53ab11d7b4b8ea6063cf157377ab0e6dc7f4cf1" cmd=["/bin/sleep","15"]
Oct 30 22:43:13.082166 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.082032    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:43:13.082166 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:43:13.082044    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="main" containerID="cri-o://8f36e2971a71dab9b08750f9c53ab11d7b4b8ea6063cf157377ab0e6dc7f4cf1"
Oct 30 22:43:13.082166 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.082110    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="main" containerID="cri-o://8f36e2971a71dab9b08750f9c53ab11d7b4b8ea6063cf157377ab0e6dc7f4cf1" gracePeriod=60
Oct 30 22:43:13.299163 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.299099    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="llm-d-routing-sidecar" containerID="cri-o://bfebe705d0eddf17a615c41df8fb538dcd3cd2fe10dd41f2d71090416337b5b8" gracePeriod=60
Oct 30 22:43:13.405275 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.405222    5664 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="main" probeResult="failure" output="Get \"https://10.134.0.27:8001/health\": dial tcp 10.134.0.27:8001: connect: connection refused"
Oct 30 22:43:13.534304 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.534270    5664 generic.go:356] "Generic (PLEG): container finished" podID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerID="8f36e2971a71dab9b08750f9c53ab11d7b4b8ea6063cf157377ab0e6dc7f4cf1" exitCode=0
Oct 30 22:43:13.534304 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.534302    5664 generic.go:356] "Generic (PLEG): container finished" podID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerID="bfebe705d0eddf17a615c41df8fb538dcd3cd2fe10dd41f2d71090416337b5b8" exitCode=0
Oct 30 22:43:13.534483 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.534349    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" event={"ID":"b577cf28-c807-4d34-9f7e-af47e6f81c48","Type":"ContainerDied","Data":"8f36e2971a71dab9b08750f9c53ab11d7b4b8ea6063cf157377ab0e6dc7f4cf1"}
Oct 30 22:43:13.534483 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.534410    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" event={"ID":"b577cf28-c807-4d34-9f7e-af47e6f81c48","Type":"ContainerDied","Data":"bfebe705d0eddf17a615c41df8fb538dcd3cd2fe10dd41f2d71090416337b5b8"}
Oct 30 22:43:13.534483 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.534427    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g" event={"ID":"b577cf28-c807-4d34-9f7e-af47e6f81c48","Type":"ContainerDied","Data":"853b46acf4eb578d306f1106025cdf434fefa03ff1551cda3983c4da2f5746bd"}
Oct 30 22:43:13.534483 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.534445    5664 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="853b46acf4eb578d306f1106025cdf434fefa03ff1551cda3983c4da2f5746bd"
Oct 30 22:43:13.547145 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.547121    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:43:13.633260 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633223    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-tmp-dir\") pod \"b577cf28-c807-4d34-9f7e-af47e6f81c48\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") "
Oct 30 22:43:13.633260 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633259    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-dshm\") pod \"b577cf28-c807-4d34-9f7e-af47e6f81c48\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") "
Oct 30 22:43:13.633487 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633344    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-home\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-home\") pod \"b577cf28-c807-4d34-9f7e-af47e6f81c48\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") "
Oct 30 22:43:13.633487 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633372    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b577cf28-c807-4d34-9f7e-af47e6f81c48-kube-api-access-2s6gh\" (UniqueName: \"kubernetes.io/projected/b577cf28-c807-4d34-9f7e-af47e6f81c48-kube-api-access-2s6gh\") pod \"b577cf28-c807-4d34-9f7e-af47e6f81c48\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") "
Oct 30 22:43:13.633487 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633409    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b577cf28-c807-4d34-9f7e-af47e6f81c48-tls-certs\" (UniqueName: \"kubernetes.io/secret/b577cf28-c807-4d34-9f7e-af47e6f81c48-tls-certs\") pod \"b577cf28-c807-4d34-9f7e-af47e6f81c48\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") "
Oct 30 22:43:13.633487 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633435    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-model-cache\") pod \"b577cf28-c807-4d34-9f7e-af47e6f81c48\" (UID: \"b577cf28-c807-4d34-9f7e-af47e6f81c48\") "
Oct 30 22:43:13.633662 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633571    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-tmp-dir" pod "b577cf28-c807-4d34-9f7e-af47e6f81c48" (UID: "b577cf28-c807-4d34-9f7e-af47e6f81c48"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:43:13.633662 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633638    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-home" pod "b577cf28-c807-4d34-9f7e-af47e6f81c48" (UID: "b577cf28-c807-4d34-9f7e-af47e6f81c48"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:43:13.633770 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633730    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.633826 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633769    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.633878 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.633818    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-model-cache" pod "b577cf28-c807-4d34-9f7e-af47e6f81c48" (UID: "b577cf28-c807-4d34-9f7e-af47e6f81c48"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:43:13.635823 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.635799    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-dshm" pod "b577cf28-c807-4d34-9f7e-af47e6f81c48" (UID: "b577cf28-c807-4d34-9f7e-af47e6f81c48"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:43:13.635823 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.635813    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b577cf28-c807-4d34-9f7e-af47e6f81c48-tls-certs" pod "b577cf28-c807-4d34-9f7e-af47e6f81c48" (UID: "b577cf28-c807-4d34-9f7e-af47e6f81c48"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:43:13.636014 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.635843    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b577cf28-c807-4d34-9f7e-af47e6f81c48-kube-api-access-2s6gh" pod "b577cf28-c807-4d34-9f7e-af47e6f81c48" (UID: "b577cf28-c807-4d34-9f7e-af47e6f81c48"). InnerVolumeSpecName "kube-api-access-2s6gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:43:13.735000 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.734868    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2s6gh\" (UniqueName: \"kubernetes.io/projected/b577cf28-c807-4d34-9f7e-af47e6f81c48-kube-api-access-2s6gh\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.735000 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.734944    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b577cf28-c807-4d34-9f7e-af47e6f81c48-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.735000 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.734956    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:13.735000 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:13.734965    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b577cf28-c807-4d34-9f7e-af47e6f81c48-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:43:14.538431 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:14.538402    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"
Oct 30 22:43:14.560672 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:14.560636    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"]
Oct 30 22:43:14.565725 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:14.565697    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-8468d8f4d9-tm44g"]
Oct 30 22:43:15.867851 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:43:15.867815    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" path="/var/lib/kubelet/pods/b577cf28-c807-4d34-9f7e-af47e6f81c48/volumes"
Oct 30 22:43:37.515417 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:43:37.515380    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:44:05.890011 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.889974    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"]
Oct 30 22:44:05.890557 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.890535    5664 cpu_manager.go:414] "containerMap: removing container" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="llm-d-routing-sidecar"
Oct 30 22:44:05.890651 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.890559    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="llm-d-routing-sidecar"
Oct 30 22:44:05.890651 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.890608    5664 cpu_manager.go:414] "containerMap: removing container" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="main"
Oct 30 22:44:05.890935 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.890891    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="main"
Oct 30 22:44:05.891007 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.890997    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="llm-d-routing-sidecar"
Oct 30 22:44:05.891045 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.891010    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="b577cf28-c807-4d34-9f7e-af47e6f81c48" containerName="main"
Oct 30 22:44:05.893952 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.893935    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:05.899494 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.899474    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-self-signed-certs\""
Oct 30 22:44:05.902297 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.902277    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"scheduler-v06-threshold-migration-test-kserve-dockercfg-2lrr5\""
Oct 30 22:44:05.913465 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.913443    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"openshift-service-ca.crt\""
Oct 30 22:44:05.913565 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.913530    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"kube-root-ca.crt\""
Oct 30 22:44:05.916492 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.916467    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"]
Oct 30 22:44:05.932521 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.932491    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-40b85f95\"/\"default-dockercfg-b4s4l\""
Oct 30 22:44:05.984320 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.984285    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:05.984492 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.984333    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:05.984492 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.984436    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-home\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:05.984492 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.984466    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc85c\" (UniqueName: \"kubernetes.io/projected/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-kube-api-access-tc85c\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:05.984599 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.984491    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:05.984599 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:05.984523    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.085670 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.085632    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.085670 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.085670    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-home\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.085956 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.085688    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc85c\" (UniqueName: \"kubernetes.io/projected/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-kube-api-access-tc85c\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.085956 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.085708    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.085956 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.085727    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.085956 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.085823    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.086204 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.086181    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-model-cache\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.086256 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.086206    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-home\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.086256 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.086238    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tmp-dir\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.088094 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.088073    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-dshm\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.088370 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.088348    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tls-certs\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.099747 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.099724    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc85c\" (UniqueName: \"kubernetes.io/projected/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-kube-api-access-tc85c\") pod \"scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") " pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.207074 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.207044    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:06.487161 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.487087    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"]
Oct 30 22:44:06.490479 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:44:06.490443    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6d3b5e_4c83_44a3_8e51_70b100d83a77.slice/crio-7047ec67fc58f4b5909632854736e6d77b643ecb8fa3a4256491bd88a5187a90 WatchSource:0}: Error finding container 7047ec67fc58f4b5909632854736e6d77b643ecb8fa3a4256491bd88a5187a90: Status 404 returned error can't find the container with id 7047ec67fc58f4b5909632854736e6d77b643ecb8fa3a4256491bd88a5187a90
Oct 30 22:44:06.780604 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.780485    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" event={"ID":"2d6d3b5e-4c83-44a3-8e51-70b100d83a77","Type":"ContainerStarted","Data":"a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb"}
Oct 30 22:44:06.780912 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.780860    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" event={"ID":"2d6d3b5e-4c83-44a3-8e51-70b100d83a77","Type":"ContainerStarted","Data":"7047ec67fc58f4b5909632854736e6d77b643ecb8fa3a4256491bd88a5187a90"}
Oct 30 22:44:06.781650 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:06.781614    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:07.572062 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:44:07.572025    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:44:07.815153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:07.815101    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" event={"ID":"2d6d3b5e-4c83-44a3-8e51-70b100d83a77","Type":"ContainerStarted","Data":"70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46"}
Oct 30 22:44:07.858083 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:07.857967    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" podStartSLOduration=2.857947122 podStartE2EDuration="2.857947122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:44:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:44:07.850675685 +0000 UTC m=+1478.102139153" watchObservedRunningTime="2026-10-30 22:44:07.857947122 +0000 UTC m=+1478.109410591"
Oct 30 22:44:08.218048 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:08.218015    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:16.207482 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:16.207444    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:16.220628 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:16.220602    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:16.225704 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:16.225667    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:16.888607 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:16.888578    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:37.515073 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:44:37.515027    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:44:41.254464 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.254434    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"]
Oct 30 22:44:41.263943 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:44:41.263888    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46" cmd=["/bin/sleep","15"]
Oct 30 22:44:41.264095 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.263958    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:44:41.264095 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:44:41.263972    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerName="main" containerID="cri-o://70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46"
Oct 30 22:44:41.264095 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.264029    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerName="main" containerID="cri-o://70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46" gracePeriod=60
Oct 30 22:44:41.481238 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.481183    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerName="llm-d-routing-sidecar" containerID="cri-o://a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb" gracePeriod=60
Oct 30 22:44:41.719226 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.719202    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:41.854290 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854204    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tmp-dir\") pod \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") "
Oct 30 22:44:41.854290 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854251    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-model-cache\") pod \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") "
Oct 30 22:44:41.854504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854293    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tls-certs\" (UniqueName: \"kubernetes.io/secret/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tls-certs\") pod \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") "
Oct 30 22:44:41.854504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854345    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-kube-api-access-tc85c\" (UniqueName: \"kubernetes.io/projected/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-kube-api-access-tc85c\") pod \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") "
Oct 30 22:44:41.854504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854398    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-home\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-home\") pod \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") "
Oct 30 22:44:41.854504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854449    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-dshm\") pod \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\" (UID: \"2d6d3b5e-4c83-44a3-8e51-70b100d83a77\") "
Oct 30 22:44:41.854721 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854526    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tmp-dir" pod "2d6d3b5e-4c83-44a3-8e51-70b100d83a77" (UID: "2d6d3b5e-4c83-44a3-8e51-70b100d83a77"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:44:41.854721 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854597    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-model-cache" pod "2d6d3b5e-4c83-44a3-8e51-70b100d83a77" (UID: "2d6d3b5e-4c83-44a3-8e51-70b100d83a77"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:44:41.854721 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854708    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-home" pod "2d6d3b5e-4c83-44a3-8e51-70b100d83a77" (UID: "2d6d3b5e-4c83-44a3-8e51-70b100d83a77"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:44:41.854860 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854767    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:44:41.854860 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.854783    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:44:41.856700 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.856668    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-dshm" pod "2d6d3b5e-4c83-44a3-8e51-70b100d83a77" (UID: "2d6d3b5e-4c83-44a3-8e51-70b100d83a77"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:44:41.856807 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.856755    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tls-certs" pod "2d6d3b5e-4c83-44a3-8e51-70b100d83a77" (UID: "2d6d3b5e-4c83-44a3-8e51-70b100d83a77"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:44:41.856807 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.856748    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-kube-api-access-tc85c" pod "2d6d3b5e-4c83-44a3-8e51-70b100d83a77" (UID: "2d6d3b5e-4c83-44a3-8e51-70b100d83a77"). InnerVolumeSpecName "kube-api-access-tc85c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:44:41.955514 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.955483    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:44:41.955514 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.955511    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc85c\" (UniqueName: \"kubernetes.io/projected/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-kube-api-access-tc85c\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:44:41.955514 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.955521    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:44:41.955750 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.955529    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d6d3b5e-4c83-44a3-8e51-70b100d83a77-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:44:41.982082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.982039    5664 generic.go:356] "Generic (PLEG): container finished" podID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerID="70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46" exitCode=0
Oct 30 22:44:41.982082 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.982067    5664 generic.go:356] "Generic (PLEG): container finished" podID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerID="a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb" exitCode=0
Oct 30 22:44:41.982272 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.982097    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"
Oct 30 22:44:41.982272 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.982106    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" event={"ID":"2d6d3b5e-4c83-44a3-8e51-70b100d83a77","Type":"ContainerDied","Data":"70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46"}
Oct 30 22:44:41.982272 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.982169    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" event={"ID":"2d6d3b5e-4c83-44a3-8e51-70b100d83a77","Type":"ContainerDied","Data":"a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb"}
Oct 30 22:44:41.982272 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.982187    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6" event={"ID":"2d6d3b5e-4c83-44a3-8e51-70b100d83a77","Type":"ContainerDied","Data":"7047ec67fc58f4b5909632854736e6d77b643ecb8fa3a4256491bd88a5187a90"}
Oct 30 22:44:41.982272 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.982207    5664 scope.go:122] "RemoveContainer" containerID="70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46"
Oct 30 22:44:41.991259 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.991240    5664 scope.go:122] "RemoveContainer" containerID="a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb"
Oct 30 22:44:41.998428 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.998408    5664 scope.go:122] "RemoveContainer" containerID="70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46"
Oct 30 22:44:41.998684 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:44:41.998666    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46\": container with ID starting with 70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46 not found: ID does not exist" containerID="70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46"
Oct 30 22:44:41.998745 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.998693    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46"} err="failed to get container status \"70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46\": rpc error: code = NotFound desc = could not find container \"70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46\": container with ID starting with 70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46 not found: ID does not exist"
Oct 30 22:44:41.998745 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.998716    5664 scope.go:122] "RemoveContainer" containerID="a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb"
Oct 30 22:44:41.998972 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:44:41.998954    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb\": container with ID starting with a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb not found: ID does not exist" containerID="a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb"
Oct 30 22:44:41.999040 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.998981    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb"} err="failed to get container status \"a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb\": rpc error: code = NotFound desc = could not find container \"a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb\": container with ID starting with a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb not found: ID does not exist"
Oct 30 22:44:41.999040 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.998998    5664 scope.go:122] "RemoveContainer" containerID="70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46"
Oct 30 22:44:41.999250 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.999232    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46"} err="failed to get container status \"70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46\": rpc error: code = NotFound desc = could not find container \"70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46\": container with ID starting with 70bd5471dd18d20b556a234738a9bb32f5a4a0583b44ce53fef89a8e2d33db46 not found: ID does not exist"
Oct 30 22:44:41.999287 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.999250    5664 scope.go:122] "RemoveContainer" containerID="a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb"
Oct 30 22:44:41.999448 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:41.999429    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb"} err="failed to get container status \"a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb\": rpc error: code = NotFound desc = could not find container \"a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb\": container with ID starting with a56f351efac195d4da4f67c35bb1c2f8b8f4137092b99b792f5828d8fef658bb not found: ID does not exist"
Oct 30 22:44:42.003311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:42.003290    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"]
Oct 30 22:44:42.009811 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:42.009783    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-40b85f95/scheduler-v06-threshold-migration-test-kserve-bc5ffbf5b-qdsg6"]
Oct 30 22:44:43.867845 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:44:43.867809    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" path="/var/lib/kubelet/pods/2d6d3b5e-4c83-44a3-8e51-70b100d83a77/volumes"
Oct 30 22:45:07.515479 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:45:07.515443    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:45:33.761226 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.761190    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"]
Oct 30 22:45:33.761993 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.761970    5664 cpu_manager.go:414] "containerMap: removing container" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerName="llm-d-routing-sidecar"
Oct 30 22:45:33.761993 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.761990    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerName="llm-d-routing-sidecar"
Oct 30 22:45:33.762153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.762016    5664 cpu_manager.go:414] "containerMap: removing container" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerName="main"
Oct 30 22:45:33.762153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.762021    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerName="main"
Oct 30 22:45:33.762153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.762071    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerName="llm-d-routing-sidecar"
Oct 30 22:45:33.762153 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.762083    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d6d3b5e-4c83-44a3-8e51-70b100d83a77" containerName="main"
Oct 30 22:45:33.766409 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.766387    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.785110 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.785083    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"]
Oct 30 22:45:33.856816 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.856787    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Oct 30 22:45:33.857022 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.856817    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"seaweedfs-s3-creds\""
Oct 30 22:45:33.857022 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.856835    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-jwx64\""
Oct 30 22:45:33.860077 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.860056    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Oct 30 22:45:33.860208 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.860116    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:45:33.860208 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.860122    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Oct 30 22:45:33.892598 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.892566    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/14ac591e-e572-4245-8a99-bfd4e86ad24b-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.892598 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.892597    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g49b5\" (UniqueName: \"kubernetes.io/projected/14ac591e-e572-4245-8a99-bfd4e86ad24b-kube-api-access-g49b5\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.892812 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.892624    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14ac591e-e572-4245-8a99-bfd4e86ad24b-tls-certs\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.892812 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.892748    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.892812 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.892786    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-model-cache\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.892812 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.892809    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-home\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.892992 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.892839    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-dshm\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.892992 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.892916    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.993634 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.993595    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/14ac591e-e572-4245-8a99-bfd4e86ad24b-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.993634 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.993631    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g49b5\" (UniqueName: \"kubernetes.io/projected/14ac591e-e572-4245-8a99-bfd4e86ad24b-kube-api-access-g49b5\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.993840 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.993658    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14ac591e-e572-4245-8a99-bfd4e86ad24b-tls-certs\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.993840 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.993783    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.993967 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.993843    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-model-cache\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.993967 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.993874    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-home\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.993967 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.993928    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-dshm\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.994120 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.993978    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.994223 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.994202    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.994334 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.994310    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.994334 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.994324    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-model-cache\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.994448 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.994430    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/14ac591e-e572-4245-8a99-bfd4e86ad24b-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.994526 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.994410    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-home\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.996258 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.996234    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-dshm\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:33.996386 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:33.996369    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14ac591e-e572-4245-8a99-bfd4e86ad24b-tls-certs\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:34.020665 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:34.020604    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g49b5\" (UniqueName: \"kubernetes.io/projected/14ac591e-e572-4245-8a99-bfd4e86ad24b-kube-api-access-g49b5\") pod \"tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:34.080586 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:34.080543    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:34.369478 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:34.369391    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"]
Oct 30 22:45:34.373228 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:45:34.373201    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14ac591e_e572_4245_8a99_bfd4e86ad24b.slice/crio-4341203a478fc8f4f1c2c7387a5018b68688c83d6dcd5e1c65a501aca69b301d WatchSource:0}: Error finding container 4341203a478fc8f4f1c2c7387a5018b68688c83d6dcd5e1c65a501aca69b301d: Status 404 returned error can't find the container with id 4341203a478fc8f4f1c2c7387a5018b68688c83d6dcd5e1c65a501aca69b301d
Oct 30 22:45:34.375159 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:34.375140    5664 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:45:35.210834 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:35.210793    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" event={"ID":"14ac591e-e572-4245-8a99-bfd4e86ad24b","Type":"ContainerStarted","Data":"4341203a478fc8f4f1c2c7387a5018b68688c83d6dcd5e1c65a501aca69b301d"}
Oct 30 22:45:37.517348 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:45:37.517304    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:45:39.238739 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:39.238701    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" event={"ID":"14ac591e-e572-4245-8a99-bfd4e86ad24b","Type":"ContainerStarted","Data":"e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828"}
Oct 30 22:45:42.256852 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:42.256817    5664 generic.go:356] "Generic (PLEG): container finished" podID="14ac591e-e572-4245-8a99-bfd4e86ad24b" containerID="e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828" exitCode=0
Oct 30 22:45:42.257263 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:42.256922    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" event={"ID":"14ac591e-e572-4245-8a99-bfd4e86ad24b","Type":"ContainerDied","Data":"e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828"}
Oct 30 22:45:43.263043 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:43.263005    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" event={"ID":"14ac591e-e572-4245-8a99-bfd4e86ad24b","Type":"ContainerStarted","Data":"2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b"}
Oct 30 22:45:43.287101 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:43.287053    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" podStartSLOduration=3.332861655 podStartE2EDuration="10.287005762s" totalImagesPullingTime="4.779117638s" totalInitContainerRuntime="2.175026469s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:45:43.286300583 +0000 UTC m=+1573.537764057" watchObservedRunningTime="2026-10-30 22:45:43.287005762 +0000 UTC m=+1573.538469230"
Oct 30 22:45:44.081298 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:44.081256    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:44.081481 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:44.081334    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:44.093621 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:44.093595    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:45:44.277939 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:45:44.277886    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:46:07.514638 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:46:07.514573    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:46:11.746483 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:11.746449    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"]
Oct 30 22:46:11.756248 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:46:11.756208    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b" cmd=["/bin/sleep","15"]
Oct 30 22:46:11.756421 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:11.756264    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:46:11.756421 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:46:11.756277    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" podUID="14ac591e-e572-4245-8a99-bfd4e86ad24b" containerName="main" containerID="cri-o://2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b"
Oct 30 22:46:11.756421 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:11.756321    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" podUID="14ac591e-e572-4245-8a99-bfd4e86ad24b" containerName="main" containerID="cri-o://2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b" gracePeriod=60
Oct 30 22:46:12.011925 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.011877    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:46:12.134312 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134283    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-home\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-home\") pod \"14ac591e-e572-4245-8a99-bfd4e86ad24b\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") "
Oct 30 22:46:12.134504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134320    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-kserve-provision-location\") pod \"14ac591e-e572-4245-8a99-bfd4e86ad24b\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") "
Oct 30 22:46:12.134504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134365    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14ac591e-e572-4245-8a99-bfd4e86ad24b-tls-certs\" (UniqueName: \"kubernetes.io/secret/14ac591e-e572-4245-8a99-bfd4e86ad24b-tls-certs\") pod \"14ac591e-e572-4245-8a99-bfd4e86ad24b\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") "
Oct 30 22:46:12.134504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134391    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-tmp-dir\") pod \"14ac591e-e572-4245-8a99-bfd4e86ad24b\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") "
Oct 30 22:46:12.134504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134449    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-model-cache\") pod \"14ac591e-e572-4245-8a99-bfd4e86ad24b\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") "
Oct 30 22:46:12.134504 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134479    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14ac591e-e572-4245-8a99-bfd4e86ad24b-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/14ac591e-e572-4245-8a99-bfd4e86ad24b-cabundle-cert\") pod \"14ac591e-e572-4245-8a99-bfd4e86ad24b\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") "
Oct 30 22:46:12.134793 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134505    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14ac591e-e572-4245-8a99-bfd4e86ad24b-kube-api-access-g49b5\" (UniqueName: \"kubernetes.io/projected/14ac591e-e572-4245-8a99-bfd4e86ad24b-kube-api-access-g49b5\") pod \"14ac591e-e572-4245-8a99-bfd4e86ad24b\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") "
Oct 30 22:46:12.134793 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134545    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-dshm\") pod \"14ac591e-e572-4245-8a99-bfd4e86ad24b\" (UID: \"14ac591e-e572-4245-8a99-bfd4e86ad24b\") "
Oct 30 22:46:12.134793 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134655    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-home" pod "14ac591e-e572-4245-8a99-bfd4e86ad24b" (UID: "14ac591e-e572-4245-8a99-bfd4e86ad24b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:12.134793 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134747    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-tmp-dir" pod "14ac591e-e572-4245-8a99-bfd4e86ad24b" (UID: "14ac591e-e572-4245-8a99-bfd4e86ad24b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:12.135025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134858    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-model-cache" pod "14ac591e-e572-4245-8a99-bfd4e86ad24b" (UID: "14ac591e-e572-4245-8a99-bfd4e86ad24b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:12.135025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134884    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:12.135025 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.134933    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:12.135508 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.135476    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14ac591e-e572-4245-8a99-bfd4e86ad24b-cabundle-cert" pod "14ac591e-e572-4245-8a99-bfd4e86ad24b" (UID: "14ac591e-e572-4245-8a99-bfd4e86ad24b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:46:12.136996 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.136970    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14ac591e-e572-4245-8a99-bfd4e86ad24b-tls-certs" pod "14ac591e-e572-4245-8a99-bfd4e86ad24b" (UID: "14ac591e-e572-4245-8a99-bfd4e86ad24b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:46:12.137086 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.137064    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14ac591e-e572-4245-8a99-bfd4e86ad24b-kube-api-access-g49b5" pod "14ac591e-e572-4245-8a99-bfd4e86ad24b" (UID: "14ac591e-e572-4245-8a99-bfd4e86ad24b"). InnerVolumeSpecName "kube-api-access-g49b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:46:12.137086 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.137066    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-dshm" pod "14ac591e-e572-4245-8a99-bfd4e86ad24b" (UID: "14ac591e-e572-4245-8a99-bfd4e86ad24b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:12.153406 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.153374    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-kserve-provision-location" pod "14ac591e-e572-4245-8a99-bfd4e86ad24b" (UID: "14ac591e-e572-4245-8a99-bfd4e86ad24b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:46:12.236353 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.236318    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:12.236353 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.236349    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/14ac591e-e572-4245-8a99-bfd4e86ad24b-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:12.236353 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.236360    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g49b5\" (UniqueName: \"kubernetes.io/projected/14ac591e-e572-4245-8a99-bfd4e86ad24b-kube-api-access-g49b5\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:12.236565 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.236369    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:12.236565 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.236378    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/14ac591e-e572-4245-8a99-bfd4e86ad24b-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:12.236565 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.236387    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14ac591e-e572-4245-8a99-bfd4e86ad24b-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:46:12.386498 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.386408    5664 generic.go:356] "Generic (PLEG): container finished" podID="14ac591e-e572-4245-8a99-bfd4e86ad24b" containerID="2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b" exitCode=0
Oct 30 22:46:12.386498 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.386462    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"
Oct 30 22:46:12.386703 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.386501    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" event={"ID":"14ac591e-e572-4245-8a99-bfd4e86ad24b","Type":"ContainerDied","Data":"2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b"}
Oct 30 22:46:12.386703 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.386557    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz" event={"ID":"14ac591e-e572-4245-8a99-bfd4e86ad24b","Type":"ContainerDied","Data":"4341203a478fc8f4f1c2c7387a5018b68688c83d6dcd5e1c65a501aca69b301d"}
Oct 30 22:46:12.386703 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.386588    5664 scope.go:122] "RemoveContainer" containerID="2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b"
Oct 30 22:46:12.396083 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.396065    5664 scope.go:122] "RemoveContainer" containerID="e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828"
Oct 30 22:46:12.406150 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.406132    5664 scope.go:122] "RemoveContainer" containerID="2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b"
Oct 30 22:46:12.406508 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:46:12.406491    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b\": container with ID starting with 2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b not found: ID does not exist" containerID="2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b"
Oct 30 22:46:12.406603 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.406519    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b"} err="failed to get container status \"2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b\": rpc error: code = NotFound desc = could not find container \"2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b\": container with ID starting with 2a5c94e5484e7e9ecd26500845897e49a2d70aa9237b4875831ee297e9797b6b not found: ID does not exist"
Oct 30 22:46:12.406603 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.406541    5664 scope.go:122] "RemoveContainer" containerID="e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828"
Oct 30 22:46:12.406816 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:46:12.406799    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828\": container with ID starting with e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828 not found: ID does not exist" containerID="e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828"
Oct 30 22:46:12.406854 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.406822    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828"} err="failed to get container status \"e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828\": rpc error: code = NotFound desc = could not find container \"e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828\": container with ID starting with e35fbd95d438d7a7f1b49f3b76508dd3868fb27067ce4cc883668a4ae4b6f828 not found: ID does not exist"
Oct 30 22:46:12.424866 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.424838    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"]
Oct 30 22:46:12.430005 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:12.429984    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-79d7476f4b-6rsfz"]
Oct 30 22:46:13.872601 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:46:13.872566    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14ac591e-e572-4245-8a99-bfd4e86ad24b" path="/var/lib/kubelet/pods/14ac591e-e572-4245-8a99-bfd4e86ad24b/volumes"
Oct 30 22:46:37.514869 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:46:37.514834    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:47:07.514835 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:47:07.514799    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:47:08.277650 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.277618    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"]
Oct 30 22:47:08.277995 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.277981    5664 cpu_manager.go:414] "containerMap: removing container" podUID="14ac591e-e572-4245-8a99-bfd4e86ad24b" containerName="storage-initializer"
Oct 30 22:47:08.278049 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.277996    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14ac591e-e572-4245-8a99-bfd4e86ad24b" containerName="storage-initializer"
Oct 30 22:47:08.278049 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.278020    5664 cpu_manager.go:414] "containerMap: removing container" podUID="14ac591e-e572-4245-8a99-bfd4e86ad24b" containerName="main"
Oct 30 22:47:08.278049 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.278027    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14ac591e-e572-4245-8a99-bfd4e86ad24b" containerName="main"
Oct 30 22:47:08.278141 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.278078    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="14ac591e-e572-4245-8a99-bfd4e86ad24b" containerName="main"
Oct 30 22:47:08.281275 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.281251    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.290939 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.290916    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Oct 30 22:47:08.291065 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.290964    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:47:08.291065 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.290964    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"seaweedfs-s3-creds\""
Oct 30 22:47:08.291065 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.290999    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Oct 30 22:47:08.291206 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.291093    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Oct 30 22:47:08.292177 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.292164    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-9s22j\""
Oct 30 22:47:08.294870 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.294849    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"]
Oct 30 22:47:08.413558 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.413493    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2a8748e6-b596-4534-86a3-6b2e817d35bc-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.414063 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.414032    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-model-cache\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.414208 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.414087    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a8748e6-b596-4534-86a3-6b2e817d35bc-tls-certs\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.414208 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.414174    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.414324 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.414234    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-dshm\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.414324 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.414270    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.414324 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.414291    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcn6z\" (UniqueName: \"kubernetes.io/projected/2a8748e6-b596-4534-86a3-6b2e817d35bc-kube-api-access-wcn6z\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.414324 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.414316    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-home\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.515711 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.515679    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-model-cache\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.515711 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.515715    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a8748e6-b596-4534-86a3-6b2e817d35bc-tls-certs\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.516319 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.515750    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.516319 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.515784    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-dshm\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.516319 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.515808    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.516319 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.515837    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcn6z\" (UniqueName: \"kubernetes.io/projected/2a8748e6-b596-4534-86a3-6b2e817d35bc-kube-api-access-wcn6z\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.516319 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.515871    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-home\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.516319 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.515930    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2a8748e6-b596-4534-86a3-6b2e817d35bc-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.516319 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.516251    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-model-cache\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.516319 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.516268    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.518375 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.518336    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2a8748e6-b596-4534-86a3-6b2e817d35bc-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.519542 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.519513    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a8748e6-b596-4534-86a3-6b2e817d35bc-tls-certs\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.519832 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.519814    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-dshm\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.520471 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.520447    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.520594 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.520574    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-home\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.537186 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.537122    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcn6z\" (UniqueName: \"kubernetes.io/projected/2a8748e6-b596-4534-86a3-6b2e817d35bc-kube-api-access-wcn6z\") pod \"tokenizer-migration-path-test-kserve-8b767c9df-gmwsq\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.598409 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.598381    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:08.878432 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:08.878350    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"]
Oct 30 22:47:08.881239 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:47:08.881213    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a8748e6_b596_4534_86a3_6b2e817d35bc.slice/crio-ef32b073db7e9cc35f438eadeedbb0d5dbba08de38c68042e3385cd39dc428d3 WatchSource:0}: Error finding container ef32b073db7e9cc35f438eadeedbb0d5dbba08de38c68042e3385cd39dc428d3: Status 404 returned error can't find the container with id ef32b073db7e9cc35f438eadeedbb0d5dbba08de38c68042e3385cd39dc428d3
Oct 30 22:47:09.622546 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:09.622497    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" event={"ID":"2a8748e6-b596-4534-86a3-6b2e817d35bc","Type":"ContainerStarted","Data":"227bbaa7545b74501cecb208862f85d846e48345da642a4c917c0b6439c1a140"}
Oct 30 22:47:09.622979 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:09.622639    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" event={"ID":"2a8748e6-b596-4534-86a3-6b2e817d35bc","Type":"ContainerStarted","Data":"ef32b073db7e9cc35f438eadeedbb0d5dbba08de38c68042e3385cd39dc428d3"}
Oct 30 22:47:11.632433 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:11.632341    5664 generic.go:356] "Generic (PLEG): container finished" podID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerID="227bbaa7545b74501cecb208862f85d846e48345da642a4c917c0b6439c1a140" exitCode=0
Oct 30 22:47:11.632913 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:11.632420    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" event={"ID":"2a8748e6-b596-4534-86a3-6b2e817d35bc","Type":"ContainerDied","Data":"227bbaa7545b74501cecb208862f85d846e48345da642a4c917c0b6439c1a140"}
Oct 30 22:47:12.638468 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:12.638434    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" event={"ID":"2a8748e6-b596-4534-86a3-6b2e817d35bc","Type":"ContainerStarted","Data":"f9e0c65483176245b68582e4c9edd4155d1520d92d4fd1d1b5e184deff6a51d0"}
Oct 30 22:47:12.664797 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:12.664749    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" podStartSLOduration=2.35198527 podStartE2EDuration="4.664735251s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.312749981s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:47:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:47:12.663243716 +0000 UTC m=+1662.914707183" watchObservedRunningTime="2026-10-30 22:47:12.664735251 +0000 UTC m=+1662.916198718"
Oct 30 22:47:13.598914 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:13.598859    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:18.599131 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:18.599093    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:18.611471 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:18.611443    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:18.674013 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:18.673986    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:37.514409 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:47:37.514374    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:47:46.504991 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.504960    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"]
Oct 30 22:47:46.514949 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:47:46.514911    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="f9e0c65483176245b68582e4c9edd4155d1520d92d4fd1d1b5e184deff6a51d0" cmd=["/bin/sleep","15"]
Oct 30 22:47:46.515125 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.514965    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:47:46.515125 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:47:46.514977    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" podUID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerName="main" containerID="cri-o://f9e0c65483176245b68582e4c9edd4155d1520d92d4fd1d1b5e184deff6a51d0"
Oct 30 22:47:46.520680 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.515038    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" podUID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerName="main" containerID="cri-o://f9e0c65483176245b68582e4c9edd4155d1520d92d4fd1d1b5e184deff6a51d0" gracePeriod=60
Oct 30 22:47:46.663530 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.663489    5664 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" podUID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Oct 30 22:47:46.780617 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.780540    5664 generic.go:356] "Generic (PLEG): container finished" podID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerID="f9e0c65483176245b68582e4c9edd4155d1520d92d4fd1d1b5e184deff6a51d0" exitCode=0
Oct 30 22:47:46.780752 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.780615    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" event={"ID":"2a8748e6-b596-4534-86a3-6b2e817d35bc","Type":"ContainerDied","Data":"f9e0c65483176245b68582e4c9edd4155d1520d92d4fd1d1b5e184deff6a51d0"}
Oct 30 22:47:46.861696 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.861673    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:46.928483 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928451    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-kserve-provision-location\") pod \"2a8748e6-b596-4534-86a3-6b2e817d35bc\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") "
Oct 30 22:47:46.928658 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928498    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a8748e6-b596-4534-86a3-6b2e817d35bc-kube-api-access-wcn6z\" (UniqueName: \"kubernetes.io/projected/2a8748e6-b596-4534-86a3-6b2e817d35bc-kube-api-access-wcn6z\") pod \"2a8748e6-b596-4534-86a3-6b2e817d35bc\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") "
Oct 30 22:47:46.928658 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928531    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-tmp-dir\") pod \"2a8748e6-b596-4534-86a3-6b2e817d35bc\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") "
Oct 30 22:47:46.928658 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928566    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-home\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-home\") pod \"2a8748e6-b596-4534-86a3-6b2e817d35bc\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") "
Oct 30 22:47:46.928658 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928584    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a8748e6-b596-4534-86a3-6b2e817d35bc-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2a8748e6-b596-4534-86a3-6b2e817d35bc-cabundle-cert\") pod \"2a8748e6-b596-4534-86a3-6b2e817d35bc\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") "
Oct 30 22:47:46.928658 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928604    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-model-cache\") pod \"2a8748e6-b596-4534-86a3-6b2e817d35bc\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") "
Oct 30 22:47:46.928658 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928630    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2a8748e6-b596-4534-86a3-6b2e817d35bc-tls-certs\" (UniqueName: \"kubernetes.io/secret/2a8748e6-b596-4534-86a3-6b2e817d35bc-tls-certs\") pod \"2a8748e6-b596-4534-86a3-6b2e817d35bc\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") "
Oct 30 22:47:46.929009 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928663    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-dshm\") pod \"2a8748e6-b596-4534-86a3-6b2e817d35bc\" (UID: \"2a8748e6-b596-4534-86a3-6b2e817d35bc\") "
Oct 30 22:47:46.929009 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928849    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-tmp-dir" pod "2a8748e6-b596-4534-86a3-6b2e817d35bc" (UID: "2a8748e6-b596-4534-86a3-6b2e817d35bc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:46.929009 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.928923    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-home" pod "2a8748e6-b596-4534-86a3-6b2e817d35bc" (UID: "2a8748e6-b596-4534-86a3-6b2e817d35bc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:46.929157 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.929067    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a8748e6-b596-4534-86a3-6b2e817d35bc-cabundle-cert" pod "2a8748e6-b596-4534-86a3-6b2e817d35bc" (UID: "2a8748e6-b596-4534-86a3-6b2e817d35bc"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:47:46.929266 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.929186    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-model-cache" pod "2a8748e6-b596-4534-86a3-6b2e817d35bc" (UID: "2a8748e6-b596-4534-86a3-6b2e817d35bc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:46.931112 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.931083    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-dshm" pod "2a8748e6-b596-4534-86a3-6b2e817d35bc" (UID: "2a8748e6-b596-4534-86a3-6b2e817d35bc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:46.931355 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.931334    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2a8748e6-b596-4534-86a3-6b2e817d35bc-tls-certs" pod "2a8748e6-b596-4534-86a3-6b2e817d35bc" (UID: "2a8748e6-b596-4534-86a3-6b2e817d35bc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:47:46.931401 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.931339    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a8748e6-b596-4534-86a3-6b2e817d35bc-kube-api-access-wcn6z" pod "2a8748e6-b596-4534-86a3-6b2e817d35bc" (UID: "2a8748e6-b596-4534-86a3-6b2e817d35bc"). InnerVolumeSpecName "kube-api-access-wcn6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:47:46.954060 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:46.954029    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-kserve-provision-location" pod "2a8748e6-b596-4534-86a3-6b2e817d35bc" (UID: "2a8748e6-b596-4534-86a3-6b2e817d35bc"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:47:47.029500 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.029468    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcn6z\" (UniqueName: \"kubernetes.io/projected/2a8748e6-b596-4534-86a3-6b2e817d35bc-kube-api-access-wcn6z\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:47.029644 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.029514    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:47.029644 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.029524    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:47.029644 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.029533    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2a8748e6-b596-4534-86a3-6b2e817d35bc-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:47.029644 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.029541    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:47.029644 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.029549    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2a8748e6-b596-4534-86a3-6b2e817d35bc-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:47.029644 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.029557    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:47.029644 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.029566    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2a8748e6-b596-4534-86a3-6b2e817d35bc-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:47:47.789913 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.789856    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq" event={"ID":"2a8748e6-b596-4534-86a3-6b2e817d35bc","Type":"ContainerDied","Data":"ef32b073db7e9cc35f438eadeedbb0d5dbba08de38c68042e3385cd39dc428d3"}
Oct 30 22:47:47.790400 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.789872    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"
Oct 30 22:47:47.790400 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.790005    5664 scope.go:122] "RemoveContainer" containerID="f9e0c65483176245b68582e4c9edd4155d1520d92d4fd1d1b5e184deff6a51d0"
Oct 30 22:47:47.803228 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.803208    5664 scope.go:122] "RemoveContainer" containerID="227bbaa7545b74501cecb208862f85d846e48345da642a4c917c0b6439c1a140"
Oct 30 22:47:47.814645 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.814601    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"]
Oct 30 22:47:47.820259 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.820238    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-8b767c9df-gmwsq"]
Oct 30 22:47:47.867853 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:47:47.867805    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a8748e6-b596-4534-86a3-6b2e817d35bc" path="/var/lib/kubelet/pods/2a8748e6-b596-4534-86a3-6b2e817d35bc/volumes"
Oct 30 22:48:07.515078 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:48:07.515038    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:48:37.515158 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:48:37.515115    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:48:38.032478 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.032447    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"]
Oct 30 22:48:38.032742 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.032731    5664 cpu_manager.go:414] "containerMap: removing container" podUID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerName="storage-initializer"
Oct 30 22:48:38.032787 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.032743    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerName="storage-initializer"
Oct 30 22:48:38.032787 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.032753    5664 cpu_manager.go:414] "containerMap: removing container" podUID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerName="main"
Oct 30 22:48:38.032787 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.032758    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerName="main"
Oct 30 22:48:38.032871 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.032830    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a8748e6-b596-4534-86a3-6b2e817d35bc" containerName="main"
Oct 30 22:48:38.037138 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.037120    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.041864 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.041840    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"seaweedfs-s3-creds\""
Oct 30 22:48:38.041996 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.041932    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-5f72j\""
Oct 30 22:48:38.043835 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.043813    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\""
Oct 30 22:48:38.047807 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.047785    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"]
Oct 30 22:48:38.120238 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.120210    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\""
Oct 30 22:48:38.120406 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.120289    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:48:38.120487 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.120453    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\""
Oct 30 22:48:38.153075 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.153040    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-home\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.153239 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.153097    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-tmp-dir\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.153288 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.153245    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-dshm\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.153322 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.153290    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ab836086-2233-429f-82f5-4670bd33a424-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.153359 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.153336    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab836086-2233-429f-82f5-4670bd33a424-tls-certs\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.153393 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.153356    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.153438 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.153395    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-model-cache\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.153515 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.153494    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt5dx\" (UniqueName: \"kubernetes.io/projected/ab836086-2233-429f-82f5-4670bd33a424-kube-api-access-xt5dx\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.253942 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.253881    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-home\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254127 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.253957    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-tmp-dir\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254127 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.253992    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-dshm\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254127 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254026    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ab836086-2233-429f-82f5-4670bd33a424-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254127 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254059    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab836086-2233-429f-82f5-4670bd33a424-tls-certs\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254339 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254165    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254339 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254237    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-model-cache\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254339 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254291    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt5dx\" (UniqueName: \"kubernetes.io/projected/ab836086-2233-429f-82f5-4670bd33a424-kube-api-access-xt5dx\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254503 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254363    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-home\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254503 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254396    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-tmp-dir\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254641 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254548    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254641 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254584    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-model-cache\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.254835 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.254815    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ab836086-2233-429f-82f5-4670bd33a424-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.256654 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.256632    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-dshm\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.256715 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.256701    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab836086-2233-429f-82f5-4670bd33a424-tls-certs\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.271188 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.271166    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt5dx\" (UniqueName: \"kubernetes.io/projected/ab836086-2233-429f-82f5-4670bd33a424-kube-api-access-xt5dx\") pod \"tokenizer-template-args-test-kserve-6784ddf685-rrhw6\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.353525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.353444    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:38.633100 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.633030    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"]
Oct 30 22:48:38.637726 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:48:38.637681    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab836086_2233_429f_82f5_4670bd33a424.slice/crio-31c93942491e35605a472e5bd24d442d597d306d9a053bb6e81c714c446155ab WatchSource:0}: Error finding container 31c93942491e35605a472e5bd24d442d597d306d9a053bb6e81c714c446155ab: Status 404 returned error can't find the container with id 31c93942491e35605a472e5bd24d442d597d306d9a053bb6e81c714c446155ab
Oct 30 22:48:38.990352 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.990319    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" event={"ID":"ab836086-2233-429f-82f5-4670bd33a424","Type":"ContainerStarted","Data":"60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560"}
Oct 30 22:48:38.990525 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:38.990367    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" event={"ID":"ab836086-2233-429f-82f5-4670bd33a424","Type":"ContainerStarted","Data":"31c93942491e35605a472e5bd24d442d597d306d9a053bb6e81c714c446155ab"}
Oct 30 22:48:43.048004 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:43.047954    5664 generic.go:356] "Generic (PLEG): container finished" podID="ab836086-2233-429f-82f5-4670bd33a424" containerID="60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560" exitCode=0
Oct 30 22:48:43.048836 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:43.048808    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" event={"ID":"ab836086-2233-429f-82f5-4670bd33a424","Type":"ContainerDied","Data":"60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560"}
Oct 30 22:48:44.057938 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:44.057853    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" event={"ID":"ab836086-2233-429f-82f5-4670bd33a424","Type":"ContainerStarted","Data":"567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f"}
Oct 30 22:48:44.354351 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:44.354251    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:48.353978 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:48.353937    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:48.366707 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:48.366649    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:48:48.392045 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:48.391996    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" podStartSLOduration=6.730779361 podStartE2EDuration="10.391984387s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.661205026s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:48:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:48:44.09403142 +0000 UTC m=+1754.345494887" watchObservedRunningTime="2026-10-30 22:48:48.391984387 +0000 UTC m=+1758.643447854"
Oct 30 22:48:49.089186 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:48:49.089154    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:49:07.515242 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:49:07.515202    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:49:17.214776 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.214746    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"]
Oct 30 22:49:17.225538 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:49:17.225494    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f" cmd=["/bin/sleep","15"]
Oct 30 22:49:17.225683 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.225561    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:49:17.225683 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:49:17.225581    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" podUID="ab836086-2233-429f-82f5-4670bd33a424" containerName="main" containerID="cri-o://567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f"
Oct 30 22:49:17.225683 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.225638    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" podUID="ab836086-2233-429f-82f5-4670bd33a424" containerName="main" containerID="cri-o://567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f" gracePeriod=60
Oct 30 22:49:17.485536 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.485499    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:49:17.550038 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550007    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-kserve-provision-location\") pod \"ab836086-2233-429f-82f5-4670bd33a424\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") "
Oct 30 22:49:17.550038 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550046    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-model-cache\") pod \"ab836086-2233-429f-82f5-4670bd33a424\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") "
Oct 30 22:49:17.550297 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550111    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-tmp-dir\") pod \"ab836086-2233-429f-82f5-4670bd33a424\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") "
Oct 30 22:49:17.550297 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550155    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ab836086-2233-429f-82f5-4670bd33a424-tls-certs\" (UniqueName: \"kubernetes.io/secret/ab836086-2233-429f-82f5-4670bd33a424-tls-certs\") pod \"ab836086-2233-429f-82f5-4670bd33a424\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") "
Oct 30 22:49:17.550297 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550194    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-dshm\") pod \"ab836086-2233-429f-82f5-4670bd33a424\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") "
Oct 30 22:49:17.550297 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550220    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab836086-2233-429f-82f5-4670bd33a424-kube-api-access-xt5dx\" (UniqueName: \"kubernetes.io/projected/ab836086-2233-429f-82f5-4670bd33a424-kube-api-access-xt5dx\") pod \"ab836086-2233-429f-82f5-4670bd33a424\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") "
Oct 30 22:49:17.550297 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550255    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab836086-2233-429f-82f5-4670bd33a424-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ab836086-2233-429f-82f5-4670bd33a424-cabundle-cert\") pod \"ab836086-2233-429f-82f5-4670bd33a424\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") "
Oct 30 22:49:17.550297 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550291    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-home\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-home\") pod \"ab836086-2233-429f-82f5-4670bd33a424\" (UID: \"ab836086-2233-429f-82f5-4670bd33a424\") "
Oct 30 22:49:17.550627 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550347    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-tmp-dir" pod "ab836086-2233-429f-82f5-4670bd33a424" (UID: "ab836086-2233-429f-82f5-4670bd33a424"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:49:17.550627 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550356    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-model-cache" pod "ab836086-2233-429f-82f5-4670bd33a424" (UID: "ab836086-2233-429f-82f5-4670bd33a424"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:49:17.550627 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550595    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:49:17.550789 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550632    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:49:17.550789 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550649    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-home" pod "ab836086-2233-429f-82f5-4670bd33a424" (UID: "ab836086-2233-429f-82f5-4670bd33a424"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:49:17.550789 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.550759    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab836086-2233-429f-82f5-4670bd33a424-cabundle-cert" pod "ab836086-2233-429f-82f5-4670bd33a424" (UID: "ab836086-2233-429f-82f5-4670bd33a424"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:49:17.552777 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.552747    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab836086-2233-429f-82f5-4670bd33a424-kube-api-access-xt5dx" pod "ab836086-2233-429f-82f5-4670bd33a424" (UID: "ab836086-2233-429f-82f5-4670bd33a424"). InnerVolumeSpecName "kube-api-access-xt5dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:49:17.552962 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.552843    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab836086-2233-429f-82f5-4670bd33a424-tls-certs" pod "ab836086-2233-429f-82f5-4670bd33a424" (UID: "ab836086-2233-429f-82f5-4670bd33a424"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:49:17.553142 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.553120    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-dshm" pod "ab836086-2233-429f-82f5-4670bd33a424" (UID: "ab836086-2233-429f-82f5-4670bd33a424"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:49:17.569865 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.569842    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-kserve-provision-location" pod "ab836086-2233-429f-82f5-4670bd33a424" (UID: "ab836086-2233-429f-82f5-4670bd33a424"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:49:17.651038 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.650998    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab836086-2233-429f-82f5-4670bd33a424-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:49:17.651038 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.651027    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:49:17.651038 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.651037    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt5dx\" (UniqueName: \"kubernetes.io/projected/ab836086-2233-429f-82f5-4670bd33a424-kube-api-access-xt5dx\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:49:17.651038 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.651047    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ab836086-2233-429f-82f5-4670bd33a424-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:49:17.651311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.651057    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:49:17.651311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:17.651068    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ab836086-2233-429f-82f5-4670bd33a424-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:49:18.196648 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.196611    5664 generic.go:356] "Generic (PLEG): container finished" podID="ab836086-2233-429f-82f5-4670bd33a424" containerID="567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f" exitCode=0
Oct 30 22:49:18.196927 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.196677    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"
Oct 30 22:49:18.196927 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.196696    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" event={"ID":"ab836086-2233-429f-82f5-4670bd33a424","Type":"ContainerDied","Data":"567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f"}
Oct 30 22:49:18.196927 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.196756    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6" event={"ID":"ab836086-2233-429f-82f5-4670bd33a424","Type":"ContainerDied","Data":"31c93942491e35605a472e5bd24d442d597d306d9a053bb6e81c714c446155ab"}
Oct 30 22:49:18.196927 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.196780    5664 scope.go:122] "RemoveContainer" containerID="567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f"
Oct 30 22:49:18.205253 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.205236    5664 scope.go:122] "RemoveContainer" containerID="60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560"
Oct 30 22:49:18.212701 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.212683    5664 scope.go:122] "RemoveContainer" containerID="567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f"
Oct 30 22:49:18.213012 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:49:18.212994    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f\": container with ID starting with 567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f not found: ID does not exist" containerID="567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f"
Oct 30 22:49:18.213074 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.213022    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f"} err="failed to get container status \"567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f\": rpc error: code = NotFound desc = could not find container \"567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f\": container with ID starting with 567ec729e9892374f7b8072830ea589b8bc5693c68191bfd37acd006ccb79a8f not found: ID does not exist"
Oct 30 22:49:18.213074 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.213038    5664 scope.go:122] "RemoveContainer" containerID="60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560"
Oct 30 22:49:18.213274 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:49:18.213256    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560\": container with ID starting with 60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560 not found: ID does not exist" containerID="60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560"
Oct 30 22:49:18.213324 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.213284    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560"} err="failed to get container status \"60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560\": rpc error: code = NotFound desc = could not find container \"60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560\": container with ID starting with 60adabbef67d7176778245262e8d4851ef280ca9f4bbf2fe6c07f356a5a8e560 not found: ID does not exist"
Oct 30 22:49:18.218074 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.218055    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"]
Oct 30 22:49:18.222065 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:18.222043    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6784ddf685-rrhw6"]
Oct 30 22:49:19.867227 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:19.867195    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab836086-2233-429f-82f5-4670bd33a424" path="/var/lib/kubelet/pods/ab836086-2233-429f-82f5-4670bd33a424/volumes"
Oct 30 22:49:30.056385 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:30.056351    5664 scope.go:122] "RemoveContainer" containerID="8f36e2971a71dab9b08750f9c53ab11d7b4b8ea6063cf157377ab0e6dc7f4cf1"
Oct 30 22:49:30.065144 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:49:30.065120    5664 scope.go:122] "RemoveContainer" containerID="bfebe705d0eddf17a615c41df8fb538dcd3cd2fe10dd41f2d71090416337b5b8"
Oct 30 22:49:37.519035 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:49:37.518995    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:50:07.515289 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:50:07.515249    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:50:34.502221 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.502181    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"]
Oct 30 22:50:34.502805 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.502616    5664 cpu_manager.go:414] "containerMap: removing container" podUID="ab836086-2233-429f-82f5-4670bd33a424" containerName="main"
Oct 30 22:50:34.502805 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.502631    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab836086-2233-429f-82f5-4670bd33a424" containerName="main"
Oct 30 22:50:34.502805 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.502663    5664 cpu_manager.go:414] "containerMap: removing container" podUID="ab836086-2233-429f-82f5-4670bd33a424" containerName="storage-initializer"
Oct 30 22:50:34.502805 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.502668    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab836086-2233-429f-82f5-4670bd33a424" containerName="storage-initializer"
Oct 30 22:50:34.502805 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.502721    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab836086-2233-429f-82f5-4670bd33a424" containerName="main"
Oct 30 22:50:34.506068 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.506039    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.514499 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.514475    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"openshift-service-ca.crt\""
Oct 30 22:50:34.514640 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.514528    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"seaweedfs-s3-creds\""
Oct 30 22:50:34.515532 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.515511    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:50:34.515609 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.515576    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"auth-invalid-token-test-kserve-self-signed-certs\""
Oct 30 22:50:34.515609 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.515578    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"default-dockercfg-8q7fd\""
Oct 30 22:50:34.515609 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.515597    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6\"/\"kube-root-ca.crt\""
Oct 30 22:50:34.518026 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.518002    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"]
Oct 30 22:50:34.665012 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.664978    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-model-cache\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.665191 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.665022    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-tmp-dir\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.665191 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.665088    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-home\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.665191 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.665126    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-dshm\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.665191 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.665157    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-kserve-provision-location\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.665321 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.665243    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c96235f1-befc-4d65-a3ab-9ac76187895c-cabundle-cert\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.665321 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.665275    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96235f1-befc-4d65-a3ab-9ac76187895c-tls-certs\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.665321 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.665302    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njqcc\" (UniqueName: \"kubernetes.io/projected/c96235f1-befc-4d65-a3ab-9ac76187895c-kube-api-access-njqcc\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.766754 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.766669    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-home\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.766754 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.766704    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-dshm\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.766754 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.766730    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-kserve-provision-location\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767087 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.766782    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c96235f1-befc-4d65-a3ab-9ac76187895c-cabundle-cert\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767087 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.766810    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96235f1-befc-4d65-a3ab-9ac76187895c-tls-certs\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767087 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.766944    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njqcc\" (UniqueName: \"kubernetes.io/projected/c96235f1-befc-4d65-a3ab-9ac76187895c-kube-api-access-njqcc\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767087 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.767001    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-model-cache\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767087 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.767041    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-tmp-dir\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767356 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.767166    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-kserve-provision-location\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767356 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.767200    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-home\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767995 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.767563    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c96235f1-befc-4d65-a3ab-9ac76187895c-cabundle-cert\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767995 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.767661    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-tmp-dir\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.767995 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.767954    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-model-cache\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.769269 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.769250    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-dshm\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.769610 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.769594    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96235f1-befc-4d65-a3ab-9ac76187895c-tls-certs\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.781307 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.781283    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njqcc\" (UniqueName: \"kubernetes.io/projected/c96235f1-befc-4d65-a3ab-9ac76187895c-kube-api-access-njqcc\") pod \"auth-invalid-token-test-kserve-679758fddf-z9l52\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") " pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:34.822758 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:34.822732    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:50:35.107458 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:35.107272    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"]
Oct 30 22:50:35.113282 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:50:35.113238    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc96235f1_befc_4d65_a3ab_9ac76187895c.slice/crio-c61075826170fd0c49597b1aec70ab2a694ea29fbf973609c72e96fe8683fe09 WatchSource:0}: Error finding container c61075826170fd0c49597b1aec70ab2a694ea29fbf973609c72e96fe8683fe09: Status 404 returned error can't find the container with id c61075826170fd0c49597b1aec70ab2a694ea29fbf973609c72e96fe8683fe09
Oct 30 22:50:35.544217 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:35.543273    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" event={"ID":"c96235f1-befc-4d65-a3ab-9ac76187895c","Type":"ContainerStarted","Data":"8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f"}
Oct 30 22:50:35.544217 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:35.543350    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" event={"ID":"c96235f1-befc-4d65-a3ab-9ac76187895c","Type":"ContainerStarted","Data":"c61075826170fd0c49597b1aec70ab2a694ea29fbf973609c72e96fe8683fe09"}
Oct 30 22:50:37.514202 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:50:37.514161    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:50:37.552280 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:37.552253    5664 generic.go:356] "Generic (PLEG): container finished" podID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerID="8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f" exitCode=0
Oct 30 22:50:37.552441 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:37.552323    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" event={"ID":"c96235f1-befc-4d65-a3ab-9ac76187895c","Type":"ContainerDied","Data":"8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f"}
Oct 30 22:50:37.553613 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:50:37.553599    5664 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 22:51:05.706237 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:51:05.706194    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" event={"ID":"c96235f1-befc-4d65-a3ab-9ac76187895c","Type":"ContainerStarted","Data":"d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124"}
Oct 30 22:51:05.740035 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:51:05.739978    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podStartSLOduration=1.735218897 podStartE2EDuration="31.739963773s" totalImagesPullingTime="27.791120555s" totalInitContainerRuntime="2.213624321s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:50:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:51:05.737954423 +0000 UTC m=+1895.989417893" watchObservedRunningTime="2026-10-30 22:51:05.739963773 +0000 UTC m=+1895.991427252"
Oct 30 22:51:05.823785 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:51:05.823738    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:51:07.515397 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:51:07.515278    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:51:14.823312 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:51:14.823211    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:51:14.824811 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:51:14.824782    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:51:24.823575 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:51:24.823533    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:51:34.823443 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:51:34.823403    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:51:37.514884 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:51:37.514847    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:51:44.824148 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:51:44.824089    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:51:54.823887 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:51:54.823843    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:52:04.823332 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:52:04.823286    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:52:07.517465 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:52:07.517417    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:52:14.823212 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:52:14.823171    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:52:24.823440 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:52:24.823389    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:52:34.823400 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:52:34.823360    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:52:37.515513 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:52:37.515477    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:52:44.823239 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:52:44.823182    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Oct 30 22:52:54.833309 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:52:54.833281    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:52:54.841004 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:52:54.840974    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:52:56.903383 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:52:56.903348    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"]
Oct 30 22:53:07.514929 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:53:07.514860    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:53:11.926586 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:11.926542    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main" containerID="cri-o://d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124" gracePeriod=45
Oct 30 22:53:12.672503 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.672477    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:53:12.779010 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.778912    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-tmp-dir\") pod \"c96235f1-befc-4d65-a3ab-9ac76187895c\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") "
Oct 30 22:53:12.779010 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.778967    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-dshm\") pod \"c96235f1-befc-4d65-a3ab-9ac76187895c\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") "
Oct 30 22:53:12.779010 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779012    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c96235f1-befc-4d65-a3ab-9ac76187895c-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c96235f1-befc-4d65-a3ab-9ac76187895c-cabundle-cert\") pod \"c96235f1-befc-4d65-a3ab-9ac76187895c\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") "
Oct 30 22:53:12.779300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779043    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-home\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-home\") pod \"c96235f1-befc-4d65-a3ab-9ac76187895c\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") "
Oct 30 22:53:12.779300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779192    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-kserve-provision-location\") pod \"c96235f1-befc-4d65-a3ab-9ac76187895c\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") "
Oct 30 22:53:12.779300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779261    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-model-cache\") pod \"c96235f1-befc-4d65-a3ab-9ac76187895c\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") "
Oct 30 22:53:12.779300 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779292    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c96235f1-befc-4d65-a3ab-9ac76187895c-tls-certs\" (UniqueName: \"kubernetes.io/secret/c96235f1-befc-4d65-a3ab-9ac76187895c-tls-certs\") pod \"c96235f1-befc-4d65-a3ab-9ac76187895c\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") "
Oct 30 22:53:12.779516 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779320    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c96235f1-befc-4d65-a3ab-9ac76187895c-kube-api-access-njqcc\" (UniqueName: \"kubernetes.io/projected/c96235f1-befc-4d65-a3ab-9ac76187895c-kube-api-access-njqcc\") pod \"c96235f1-befc-4d65-a3ab-9ac76187895c\" (UID: \"c96235f1-befc-4d65-a3ab-9ac76187895c\") "
Oct 30 22:53:12.779571 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779509    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c96235f1-befc-4d65-a3ab-9ac76187895c-cabundle-cert" pod "c96235f1-befc-4d65-a3ab-9ac76187895c" (UID: "c96235f1-befc-4d65-a3ab-9ac76187895c"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 22:53:12.779623 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779611    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c96235f1-befc-4d65-a3ab-9ac76187895c-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:12.779680 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779635    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-model-cache" pod "c96235f1-befc-4d65-a3ab-9ac76187895c" (UID: "c96235f1-befc-4d65-a3ab-9ac76187895c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:12.779822 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.779791    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-home" pod "c96235f1-befc-4d65-a3ab-9ac76187895c" (UID: "c96235f1-befc-4d65-a3ab-9ac76187895c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:12.781742 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.781713    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-dshm" pod "c96235f1-befc-4d65-a3ab-9ac76187895c" (UID: "c96235f1-befc-4d65-a3ab-9ac76187895c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:12.781842 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.781798    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c96235f1-befc-4d65-a3ab-9ac76187895c-kube-api-access-njqcc" pod "c96235f1-befc-4d65-a3ab-9ac76187895c" (UID: "c96235f1-befc-4d65-a3ab-9ac76187895c"). InnerVolumeSpecName "kube-api-access-njqcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:53:12.781953 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.781930    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c96235f1-befc-4d65-a3ab-9ac76187895c-tls-certs" pod "c96235f1-befc-4d65-a3ab-9ac76187895c" (UID: "c96235f1-befc-4d65-a3ab-9ac76187895c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:53:12.791246 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.791223    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-tmp-dir" pod "c96235f1-befc-4d65-a3ab-9ac76187895c" (UID: "c96235f1-befc-4d65-a3ab-9ac76187895c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:12.800947 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.800920    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-kserve-provision-location" pod "c96235f1-befc-4d65-a3ab-9ac76187895c" (UID: "c96235f1-befc-4d65-a3ab-9ac76187895c"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:53:12.880377 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.880344    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:12.880377 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.880370    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:12.880377 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.880380    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c96235f1-befc-4d65-a3ab-9ac76187895c-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:12.880584 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.880389    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njqcc\" (UniqueName: \"kubernetes.io/projected/c96235f1-befc-4d65-a3ab-9ac76187895c-kube-api-access-njqcc\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:12.880584 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.880398    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:12.880584 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.880406    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:12.880584 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:12.880413    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c96235f1-befc-4d65-a3ab-9ac76187895c-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:53:13.261763 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.261727    5664 generic.go:356] "Generic (PLEG): container finished" podID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerID="d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124" exitCode=0
Oct 30 22:53:13.262185 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.261793    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" event={"ID":"c96235f1-befc-4d65-a3ab-9ac76187895c","Type":"ContainerDied","Data":"d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124"}
Oct 30 22:53:13.262185 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.261801    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"
Oct 30 22:53:13.262185 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.261862    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52" event={"ID":"c96235f1-befc-4d65-a3ab-9ac76187895c","Type":"ContainerDied","Data":"c61075826170fd0c49597b1aec70ab2a694ea29fbf973609c72e96fe8683fe09"}
Oct 30 22:53:13.262185 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.261884    5664 scope.go:122] "RemoveContainer" containerID="d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124"
Oct 30 22:53:13.270718 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.270702    5664 scope.go:122] "RemoveContainer" containerID="8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f"
Oct 30 22:53:13.279952 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.279929    5664 scope.go:122] "RemoveContainer" containerID="d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124"
Oct 30 22:53:13.280209 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:53:13.280192    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124\": container with ID starting with d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124 not found: ID does not exist" containerID="d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124"
Oct 30 22:53:13.280293 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.280221    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124"} err="failed to get container status \"d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124\": rpc error: code = NotFound desc = could not find container \"d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124\": container with ID starting with d5b2c8f3fe7788022c1bcdcebfdd41145fb7429ac27025da676e9abe93535124 not found: ID does not exist"
Oct 30 22:53:13.280293 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.280242    5664 scope.go:122] "RemoveContainer" containerID="8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f"
Oct 30 22:53:13.280495 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:53:13.280473    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f\": container with ID starting with 8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f not found: ID does not exist" containerID="8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f"
Oct 30 22:53:13.280589 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.280495    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f"} err="failed to get container status \"8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f\": rpc error: code = NotFound desc = could not find container \"8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f\": container with ID starting with 8ce80873c7e6f8205bb16deb752731035ffab526c3ab217fe3c328f79420b53f not found: ID does not exist"
Oct 30 22:53:13.286422 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.286402    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"]
Oct 30 22:53:13.293439 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.293417    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-auth-invalid-token-rejected-e7cfa2c6/auth-invalid-token-test-kserve-679758fddf-z9l52"]
Oct 30 22:53:13.867429 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:53:13.867395    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" path="/var/lib/kubelet/pods/c96235f1-befc-4d65-a3ab-9ac76187895c/volumes"
Oct 30 22:53:37.514402 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:53:37.514361    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:54:07.514713 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:54:07.514670    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:54:34.396739 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.396659    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"]
Oct 30 22:54:34.397176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.396970    5664 cpu_manager.go:414] "containerMap: removing container" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="storage-initializer"
Oct 30 22:54:34.397176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.396981    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="storage-initializer"
Oct 30 22:54:34.397176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.397000    5664 cpu_manager.go:414] "containerMap: removing container" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main"
Oct 30 22:54:34.397176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.397005    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main"
Oct 30 22:54:34.397176 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.397066    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="c96235f1-befc-4d65-a3ab-9ac76187895c" containerName="main"
Oct 30 22:54:34.399937 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.399920    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.404428 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.404401    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-6zd5k\""
Oct 30 22:54:34.405822 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.405799    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs\""
Oct 30 22:54:34.406371 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.406001    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Oct 30 22:54:34.406371 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.406365    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Oct 30 22:54:34.412376 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.412356    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"]
Oct 30 22:54:34.448128 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.448100    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.448231 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.448135    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.448231 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.448151    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.448231 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.448174    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.448345 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.448324    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.448380 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.448363    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ztxm\" (UniqueName: \"kubernetes.io/projected/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-kube-api-access-7ztxm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.549648 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.549617    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.549829 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.549654    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.549829 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.549674    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.549829 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.549697    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.549829 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.549753    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.549829 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.549788    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ztxm\" (UniqueName: \"kubernetes.io/projected/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-kube-api-access-7ztxm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.550186 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.550165    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tmp-dir\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.550265 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.550239    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-home\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.550301 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.550247    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-model-cache\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.552065 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.552048    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-dshm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.552308 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.552289    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tls-certs\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.563039 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.563004    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ztxm\" (UniqueName: \"kubernetes.io/projected/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-kube-api-access-7ztxm\") pod \"llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.711865 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.711794    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:34.995988 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:34.995884    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"]
Oct 30 22:54:34.999133 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:54:34.999104    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b8e2142_6d23_4c62_b36f_30b5d78d72d4.slice/crio-0331b102b96a29e94ff1933e121f929fcf8fb2d208447dadc212cf2f73051b0c WatchSource:0}: Error finding container 0331b102b96a29e94ff1933e121f929fcf8fb2d208447dadc212cf2f73051b0c: Status 404 returned error can't find the container with id 0331b102b96a29e94ff1933e121f929fcf8fb2d208447dadc212cf2f73051b0c
Oct 30 22:54:35.601472 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:35.601437    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt" event={"ID":"2b8e2142-6d23-4c62-b36f-30b5d78d72d4","Type":"ContainerStarted","Data":"8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40"}
Oct 30 22:54:35.601984 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:35.601492    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt" event={"ID":"2b8e2142-6d23-4c62-b36f-30b5d78d72d4","Type":"ContainerStarted","Data":"0331b102b96a29e94ff1933e121f929fcf8fb2d208447dadc212cf2f73051b0c"}
Oct 30 22:54:35.624237 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:35.624117    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt" podStartSLOduration=1.62409609 podStartE2EDuration="1.62409609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:54:35.622848333 +0000 UTC m=+2105.874311840" watchObservedRunningTime="2026-10-30 22:54:35.62409609 +0000 UTC m=+2105.875559558"
Oct 30 22:54:35.722027 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:35.721988    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:37.515437 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:54:37.515397    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:54:44.711974 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:44.711940    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:44.725449 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:44.725427    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:54:45.653583 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:54:45.653553    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:55:07.515192 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:55:07.515151    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:55:37.515552 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:55:37.515512    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:56:07.514727 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:56:07.514686    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:56:37.520295 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:56:37.520259    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:56:37.591614 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:37.591577    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"]
Oct 30 22:56:37.600662 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:56:37.600614    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40" cmd=["/bin/sleep","15"]
Oct 30 22:56:37.600810 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:37.600679    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:56:37.600810 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:56:37.600695    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt" podUID="2b8e2142-6d23-4c62-b36f-30b5d78d72d4" containerName="main" containerID="cri-o://8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40"
Oct 30 22:56:37.600810 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:37.600738    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt" podUID="2b8e2142-6d23-4c62-b36f-30b5d78d72d4" containerName="main" containerID="cri-o://8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40" gracePeriod=60
Oct 30 22:56:37.643225 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:37.643188    5664 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt" podUID="2b8e2142-6d23-4c62-b36f-30b5d78d72d4" containerName="main" probeResult="failure" output="Get \"https://10.134.0.33:8000/health\": dial tcp 10.134.0.33:8000: connect: connection refused"
Oct 30 22:56:37.977214 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:37.977187    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:56:38.096802 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.096726    5664 generic.go:356] "Generic (PLEG): container finished" podID="2b8e2142-6d23-4c62-b36f-30b5d78d72d4" containerID="8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40" exitCode=0
Oct 30 22:56:38.096802 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.096779    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"
Oct 30 22:56:38.097023 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.096814    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt" event={"ID":"2b8e2142-6d23-4c62-b36f-30b5d78d72d4","Type":"ContainerDied","Data":"8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40"}
Oct 30 22:56:38.097023 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.096871    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt" event={"ID":"2b8e2142-6d23-4c62-b36f-30b5d78d72d4","Type":"ContainerDied","Data":"0331b102b96a29e94ff1933e121f929fcf8fb2d208447dadc212cf2f73051b0c"}
Oct 30 22:56:38.097023 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.096925    5664 scope.go:122] "RemoveContainer" containerID="8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40"
Oct 30 22:56:38.105560 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.105539    5664 scope.go:122] "RemoveContainer" containerID="8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40"
Oct 30 22:56:38.105869 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:56:38.105849    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40\": container with ID starting with 8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40 not found: ID does not exist" containerID="8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40"
Oct 30 22:56:38.105947 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.105879    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40"} err="failed to get container status \"8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40\": rpc error: code = NotFound desc = could not find container \"8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40\": container with ID starting with 8993cafcb6ea944b31e21ae35ad19da21a8e98cc2524b0a7d77b86d1eb6c1b40 not found: ID does not exist"
Oct 30 22:56:38.121150 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121123    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tls-certs\") pod \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") "
Oct 30 22:56:38.121307 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121218    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-home\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-home\") pod \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") "
Oct 30 22:56:38.121307 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121278    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-model-cache\") pod \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") "
Oct 30 22:56:38.121419 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121313    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tmp-dir\") pod \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") "
Oct 30 22:56:38.121419 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121360    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-kube-api-access-7ztxm\" (UniqueName: \"kubernetes.io/projected/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-kube-api-access-7ztxm\") pod \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") "
Oct 30 22:56:38.121515 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121443    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-dshm\") pod \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\" (UID: \"2b8e2142-6d23-4c62-b36f-30b5d78d72d4\") "
Oct 30 22:56:38.121567 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121513    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-home" pod "2b8e2142-6d23-4c62-b36f-30b5d78d72d4" (UID: "2b8e2142-6d23-4c62-b36f-30b5d78d72d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:38.121644 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121617    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tmp-dir" pod "2b8e2142-6d23-4c62-b36f-30b5d78d72d4" (UID: "2b8e2142-6d23-4c62-b36f-30b5d78d72d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:38.121816 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121757    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:38.121816 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.121781    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:38.122206 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.122186    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-model-cache" pod "2b8e2142-6d23-4c62-b36f-30b5d78d72d4" (UID: "2b8e2142-6d23-4c62-b36f-30b5d78d72d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:38.123741 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.123720    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-dshm" pod "2b8e2142-6d23-4c62-b36f-30b5d78d72d4" (UID: "2b8e2142-6d23-4c62-b36f-30b5d78d72d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:56:38.123835 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.123780    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tls-certs" pod "2b8e2142-6d23-4c62-b36f-30b5d78d72d4" (UID: "2b8e2142-6d23-4c62-b36f-30b5d78d72d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:56:38.124225 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.124208    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-kube-api-access-7ztxm" pod "2b8e2142-6d23-4c62-b36f-30b5d78d72d4" (UID: "2b8e2142-6d23-4c62-b36f-30b5d78d72d4"). InnerVolumeSpecName "kube-api-access-7ztxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:56:38.223147 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.223110    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:38.223147 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.223141    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ztxm\" (UniqueName: \"kubernetes.io/projected/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-kube-api-access-7ztxm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:38.223445 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.223159    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:38.223445 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.223168    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b8e2142-6d23-4c62-b36f-30b5d78d72d4-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:56:38.421472 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.421443    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"]
Oct 30 22:56:38.424250 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:38.424226    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-router-managed-test-llm-4b931143-kserve-f966769b4-5tvqt"]
Oct 30 22:56:39.867311 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:39.867278    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b8e2142-6d23-4c62-b36f-30b5d78d72d4" path="/var/lib/kubelet/pods/2b8e2142-6d23-4c62-b36f-30b5d78d72d4/volumes"
Oct 30 22:56:54.303020 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.302985    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"]
Oct 30 22:56:54.303445 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.303290    5664 cpu_manager.go:414] "containerMap: removing container" podUID="2b8e2142-6d23-4c62-b36f-30b5d78d72d4" containerName="main"
Oct 30 22:56:54.303445 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.303300    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b8e2142-6d23-4c62-b36f-30b5d78d72d4" containerName="main"
Oct 30 22:56:54.303445 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.303374    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b8e2142-6d23-4c62-b36f-30b5d78d72d4" containerName="main"
Oct 30 22:56:54.306429 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.306409    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.323418 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.323392    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"openshift-service-ca.crt\""
Oct 30 22:56:54.323557 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.323489    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"kube-root-ca.crt\""
Oct 30 22:56:54.323856 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.323843    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"default-dockercfg-9fqg5\""
Oct 30 22:56:54.324781 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.324764    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-6f0aa991\"/\"llmisv122f03714c5bdf915a2917fdf1262b98-kserve-self-signed-certs\""
Oct 30 22:56:54.330594 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.330550    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"]
Oct 30 22:56:54.461303 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.461269    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.461303 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.461307    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.461522 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.461332    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.461522 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.461450    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dhw5\" (UniqueName: \"kubernetes.io/projected/b99a5872-5efc-497e-8845-4d5d110bd6f1-kube-api-access-4dhw5\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.461522 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.461498    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.461616 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.461559    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b99a5872-5efc-497e-8845-4d5d110bd6f1-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.562357 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.562278    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.562357 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.562320    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.562572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.562374    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dhw5\" (UniqueName: \"kubernetes.io/projected/b99a5872-5efc-497e-8845-4d5d110bd6f1-kube-api-access-4dhw5\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.562572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.562429    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.562572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.562468    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b99a5872-5efc-497e-8845-4d5d110bd6f1-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.562572 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.562493    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.562817 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.562791    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.562881 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.562825    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-home\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.562970 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.562940    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.564952 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.564926    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-dshm\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.565214 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.565191    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b99a5872-5efc-497e-8845-4d5d110bd6f1-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.579447 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.579423    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dhw5\" (UniqueName: \"kubernetes.io/projected/b99a5872-5efc-497e-8845-4d5d110bd6f1-kube-api-access-4dhw5\") pod \"llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") " pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.619127 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.619096    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:56:54.895712 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:54.895672    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"]
Oct 30 22:56:54.898362 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:56:54.898337    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb99a5872_5efc_497e_8845_4d5d110bd6f1.slice/crio-121cac222d05ab86fd658b660c85138085c818b1929d17c4d81975e08e7b31fd WatchSource:0}: Error finding container 121cac222d05ab86fd658b660c85138085c818b1929d17c4d81975e08e7b31fd: Status 404 returned error can't find the container with id 121cac222d05ab86fd658b660c85138085c818b1929d17c4d81975e08e7b31fd
Oct 30 22:56:55.165477 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:55.165442    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz" event={"ID":"b99a5872-5efc-497e-8845-4d5d110bd6f1","Type":"ContainerStarted","Data":"05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a"}
Oct 30 22:56:55.165679 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:55.165491    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz" event={"ID":"b99a5872-5efc-497e-8845-4d5d110bd6f1","Type":"ContainerStarted","Data":"121cac222d05ab86fd658b660c85138085c818b1929d17c4d81975e08e7b31fd"}
Oct 30 22:56:55.205392 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:55.205331    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz" podStartSLOduration=1.205317269 podStartE2EDuration="1.205317269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:56:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:56:55.203485216 +0000 UTC m=+2245.454948683" watchObservedRunningTime="2026-10-30 22:56:55.205317269 +0000 UTC m=+2245.456780736"
Oct 30 22:56:55.619939 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:56:55.619838    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:57:04.619295 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:57:04.619259    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:57:04.631248 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:57:04.631223    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:57:05.216075 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:57:05.216049    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:57:07.514018 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:57:07.513973    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:57:37.566480 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:57:37.566437    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:57:46.263343 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:57:46.263294    5664 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" podUID="320724d28f4bb4eb7790f5cce736ad83" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 22:58:06.480749 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:06.480715    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"]
Oct 30 22:58:06.489476 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:58:06.489443    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a" cmd=["/bin/sleep","15"]
Oct 30 22:58:06.489600 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:06.489491    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 22:58:06.489600 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:58:06.489504    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz" podUID="b99a5872-5efc-497e-8845-4d5d110bd6f1" containerName="main" containerID="cri-o://05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a"
Oct 30 22:58:06.489600 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:06.489549    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz" podUID="b99a5872-5efc-497e-8845-4d5d110bd6f1" containerName="main" containerID="cri-o://05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a" gracePeriod=60
Oct 30 22:58:07.233822 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.233795    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:58:07.318999 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.318969    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-home\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-home\") pod \"b99a5872-5efc-497e-8845-4d5d110bd6f1\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") "
Oct 30 22:58:07.319171 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319006    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b99a5872-5efc-497e-8845-4d5d110bd6f1-tls-certs\" (UniqueName: \"kubernetes.io/secret/b99a5872-5efc-497e-8845-4d5d110bd6f1-tls-certs\") pod \"b99a5872-5efc-497e-8845-4d5d110bd6f1\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") "
Oct 30 22:58:07.319171 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319037    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-tmp-dir\") pod \"b99a5872-5efc-497e-8845-4d5d110bd6f1\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") "
Oct 30 22:58:07.319171 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319074    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-model-cache\") pod \"b99a5872-5efc-497e-8845-4d5d110bd6f1\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") "
Oct 30 22:58:07.319171 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319135    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b99a5872-5efc-497e-8845-4d5d110bd6f1-kube-api-access-4dhw5\" (UniqueName: \"kubernetes.io/projected/b99a5872-5efc-497e-8845-4d5d110bd6f1-kube-api-access-4dhw5\") pod \"b99a5872-5efc-497e-8845-4d5d110bd6f1\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") "
Oct 30 22:58:07.319171 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319166    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-dshm\") pod \"b99a5872-5efc-497e-8845-4d5d110bd6f1\" (UID: \"b99a5872-5efc-497e-8845-4d5d110bd6f1\") "
Oct 30 22:58:07.319443 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319287    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-home" pod "b99a5872-5efc-497e-8845-4d5d110bd6f1" (UID: "b99a5872-5efc-497e-8845-4d5d110bd6f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:58:07.319443 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319356    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-tmp-dir" pod "b99a5872-5efc-497e-8845-4d5d110bd6f1" (UID: "b99a5872-5efc-497e-8845-4d5d110bd6f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:58:07.319443 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319386    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-model-cache" pod "b99a5872-5efc-497e-8845-4d5d110bd6f1" (UID: "b99a5872-5efc-497e-8845-4d5d110bd6f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:58:07.319567 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319463    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:07.319567 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319482    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:07.319567 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.319498    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:07.321410 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.321384    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-dshm" pod "b99a5872-5efc-497e-8845-4d5d110bd6f1" (UID: "b99a5872-5efc-497e-8845-4d5d110bd6f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 22:58:07.321501 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.321459    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b99a5872-5efc-497e-8845-4d5d110bd6f1-kube-api-access-4dhw5" pod "b99a5872-5efc-497e-8845-4d5d110bd6f1" (UID: "b99a5872-5efc-497e-8845-4d5d110bd6f1"). InnerVolumeSpecName "kube-api-access-4dhw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 22:58:07.321501 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.321490    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b99a5872-5efc-497e-8845-4d5d110bd6f1-tls-certs" pod "b99a5872-5efc-497e-8845-4d5d110bd6f1" (UID: "b99a5872-5efc-497e-8845-4d5d110bd6f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 22:58:07.420216 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.420189    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dhw5\" (UniqueName: \"kubernetes.io/projected/b99a5872-5efc-497e-8845-4d5d110bd6f1-kube-api-access-4dhw5\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:07.420216 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.420218    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b99a5872-5efc-497e-8845-4d5d110bd6f1-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:07.420399 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.420231    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b99a5872-5efc-497e-8845-4d5d110bd6f1-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 22:58:07.478134 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.478096    5664 generic.go:356] "Generic (PLEG): container finished" podID="b99a5872-5efc-497e-8845-4d5d110bd6f1" containerID="05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a" exitCode=0
Oct 30 22:58:07.478274 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.478147    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"
Oct 30 22:58:07.478274 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.478174    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz" event={"ID":"b99a5872-5efc-497e-8845-4d5d110bd6f1","Type":"ContainerDied","Data":"05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a"}
Oct 30 22:58:07.478274 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.478221    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz" event={"ID":"b99a5872-5efc-497e-8845-4d5d110bd6f1","Type":"ContainerDied","Data":"121cac222d05ab86fd658b660c85138085c818b1929d17c4d81975e08e7b31fd"}
Oct 30 22:58:07.478274 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.478244    5664 scope.go:122] "RemoveContainer" containerID="05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a"
Oct 30 22:58:07.487485 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.487470    5664 scope.go:122] "RemoveContainer" containerID="05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a"
Oct 30 22:58:07.487776 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:58:07.487747    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a\": container with ID starting with 05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a not found: ID does not exist" containerID="05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a"
Oct 30 22:58:07.487824 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.487771    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a"} err="failed to get container status \"05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a\": rpc error: code = NotFound desc = could not find container \"05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a\": container with ID starting with 05e2c7ffb525589c8b81c1289584b65ea74e81d80a5f96684bcb77d3f003541a not found: ID does not exist"
Oct 30 22:58:07.501455 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.501430    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"]
Oct 30 22:58:07.507305 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.507280    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz"]
Oct 30 22:58:07.515097 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:58:07.515067    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:58:07.868179 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:07.868100    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b99a5872-5efc-497e-8845-4d5d110bd6f1" path="/var/lib/kubelet/pods/b99a5872-5efc-497e-8845-4d5d110bd6f1/volumes"
Oct 30 22:58:08.206218 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:08.206180    5664 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-6f0aa991/llmisvc-model-fb-opt-125m-route-e95b1dc1-kserve-5577b665bbnszlz" podUID="b99a5872-5efc-497e-8845-4d5d110bd6f1" containerName="main" probeResult="failure" output="Get \"https://10.134.0.34:8000/health\": context deadline exceeded"
Oct 30 22:58:18.415022 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.414987    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"]
Oct 30 22:58:18.415858 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.415836    5664 cpu_manager.go:414] "containerMap: removing container" podUID="b99a5872-5efc-497e-8845-4d5d110bd6f1" containerName="main"
Oct 30 22:58:18.415946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.415861    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b99a5872-5efc-497e-8845-4d5d110bd6f1" containerName="main"
Oct 30 22:58:18.415946 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.415944    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="b99a5872-5efc-497e-8845-4d5d110bd6f1" containerName="main"
Oct 30 22:58:18.420180 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.420162    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.425843 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.425815    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"seaweedfs-s3-creds\""
Oct 30 22:58:18.426012 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.425981    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\""
Oct 30 22:58:18.426012 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.425978    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-j7xwg\""
Oct 30 22:58:18.426012 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.425980    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Oct 30 22:58:18.426242 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.426043    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 22:58:18.426242 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.426064    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\""
Oct 30 22:58:18.436518 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.433573    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"]
Oct 30 22:58:18.508451 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.508415    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-home\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.508646 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.508470    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-dshm\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.508646 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.508556    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rmxg\" (UniqueName: \"kubernetes.io/projected/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kube-api-access-4rmxg\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.508646 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.508593    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.508646 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.508620    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.508790 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.508664    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.508790 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.508737    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.508790 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.508784    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610232 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610195    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610436 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610240    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-home\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610436 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610269    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-dshm\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610436 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610289    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rmxg\" (UniqueName: \"kubernetes.io/projected/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kube-api-access-4rmxg\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610436 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610310    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610436 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610326    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610436 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610340    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610436 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610385    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610845 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610711    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.610845 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.610747    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.611068 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.611045    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-home\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.611314 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.611068    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.611314 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.611128    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.612883 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.612858    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-dshm\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.613122 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.613102    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.627369 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.627336    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rmxg\" (UniqueName: \"kubernetes.io/projected/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kube-api-access-4rmxg\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:18.739196 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:18.739113    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:19.019749 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:19.019648    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"]
Oct 30 22:58:19.023019 ip-10-0-132-192 kubenswrapper[5664]: W1030 22:58:19.022992    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a3fad52_b3fa_40e7_bac9_4d7e873f866f.slice/crio-366413d5797ac74fb7ea4e472715b406d74eda58242b0b8ed81c9b58ef44d662 WatchSource:0}: Error finding container 366413d5797ac74fb7ea4e472715b406d74eda58242b0b8ed81c9b58ef44d662: Status 404 returned error can't find the container with id 366413d5797ac74fb7ea4e472715b406d74eda58242b0b8ed81c9b58ef44d662
Oct 30 22:58:19.536965 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:19.536913    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" event={"ID":"1a3fad52-b3fa-40e7-bac9-4d7e873f866f","Type":"ContainerStarted","Data":"fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982"}
Oct 30 22:58:19.537479 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:19.536985    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" event={"ID":"1a3fad52-b3fa-40e7-bac9-4d7e873f866f","Type":"ContainerStarted","Data":"366413d5797ac74fb7ea4e472715b406d74eda58242b0b8ed81c9b58ef44d662"}
Oct 30 22:58:21.546485 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:21.546447    5664 generic.go:356] "Generic (PLEG): container finished" podID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerID="fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982" exitCode=0
Oct 30 22:58:21.546921 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:21.546496    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" event={"ID":"1a3fad52-b3fa-40e7-bac9-4d7e873f866f","Type":"ContainerDied","Data":"fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982"}
Oct 30 22:58:22.553944 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:22.553889    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" event={"ID":"1a3fad52-b3fa-40e7-bac9-4d7e873f866f","Type":"ContainerStarted","Data":"0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd"}
Oct 30 22:58:22.585121 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:22.585064    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podStartSLOduration=2.278921207 podStartE2EDuration="4.585049095s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.306127888s" isStatefulPod=false podCreationTimestamp="2026-10-30 22:58:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 22:58:22.582454499 +0000 UTC m=+2332.833917964" watchObservedRunningTime="2026-10-30 22:58:22.585049095 +0000 UTC m=+2332.836512562"
Oct 30 22:58:22.739423 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:22.739386    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:28.740090 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:28.740051    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 22:58:28.741483 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:28.741455    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 22:58:37.515007 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:58:37.514961    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:58:38.740379 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:38.740339    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 22:58:48.739780 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:48.739743    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 22:58:58.740503 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:58:58.740463    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 22:59:07.515657 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:59:07.515609    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:59:08.740148 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:59:08.740105    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 22:59:18.739869 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:59:18.739827    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 22:59:28.739636 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:59:28.739590    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 22:59:37.514538 ip-10-0-132-192 kubenswrapper[5664]: E1030 22:59:37.514493    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 22:59:38.740467 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:59:38.740424    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 22:59:48.740045 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:59:48.739996    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 22:59:58.740599 ip-10-0-132-192 kubenswrapper[5664]: I1030 22:59:58.740554    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 23:00:07.516056 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:00:07.516015    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:00:08.750499 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:08.750468    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 23:00:08.758422 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:08.758388    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 23:00:13.201014 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:13.200928    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"]
Oct 30 23:00:28.223827 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:28.223769    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" containerID="cri-o://0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd" gracePeriod=45
Oct 30 23:00:28.752152 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:28.752113    5664 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/health\": dial tcp 10.134.0.35:8000: connect: connection refused"
Oct 30 23:00:28.968533 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:28.968509    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 23:00:29.036093 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036013    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-home\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-home\") pod \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") "
Oct 30 23:00:29.036237 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036097    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-dshm\") pod \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") "
Oct 30 23:00:29.036237 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036122    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kube-api-access-4rmxg\" (UniqueName: \"kubernetes.io/projected/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kube-api-access-4rmxg\") pod \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") "
Oct 30 23:00:29.036237 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036166    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-cabundle-cert\") pod \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") "
Oct 30 23:00:29.036237 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036186    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tls-certs\" (UniqueName: \"kubernetes.io/secret/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tls-certs\") pod \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") "
Oct 30 23:00:29.036237 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036203    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-model-cache\") pod \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") "
Oct 30 23:00:29.036474 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036300    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tmp-dir\") pod \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") "
Oct 30 23:00:29.036474 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036349    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kserve-provision-location\") pod \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\" (UID: \"1a3fad52-b3fa-40e7-bac9-4d7e873f866f\") "
Oct 30 23:00:29.036572 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036469    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-model-cache" pod "1a3fad52-b3fa-40e7-bac9-4d7e873f866f" (UID: "1a3fad52-b3fa-40e7-bac9-4d7e873f866f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:00:29.036572 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036545    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-cabundle-cert" pod "1a3fad52-b3fa-40e7-bac9-4d7e873f866f" (UID: "1a3fad52-b3fa-40e7-bac9-4d7e873f866f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:00:29.036716 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036683    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:29.036716 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036708    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:29.036848 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.036738    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-home" pod "1a3fad52-b3fa-40e7-bac9-4d7e873f866f" (UID: "1a3fad52-b3fa-40e7-bac9-4d7e873f866f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:00:29.038769 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.038744    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tls-certs" pod "1a3fad52-b3fa-40e7-bac9-4d7e873f866f" (UID: "1a3fad52-b3fa-40e7-bac9-4d7e873f866f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:00:29.039072 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.039041    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kube-api-access-4rmxg" pod "1a3fad52-b3fa-40e7-bac9-4d7e873f866f" (UID: "1a3fad52-b3fa-40e7-bac9-4d7e873f866f"). InnerVolumeSpecName "kube-api-access-4rmxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:00:29.039167 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.039086    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-dshm" pod "1a3fad52-b3fa-40e7-bac9-4d7e873f866f" (UID: "1a3fad52-b3fa-40e7-bac9-4d7e873f866f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:00:29.049075 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.049050    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tmp-dir" pod "1a3fad52-b3fa-40e7-bac9-4d7e873f866f" (UID: "1a3fad52-b3fa-40e7-bac9-4d7e873f866f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:00:29.057434 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.057407    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kserve-provision-location" pod "1a3fad52-b3fa-40e7-bac9-4d7e873f866f" (UID: "1a3fad52-b3fa-40e7-bac9-4d7e873f866f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:00:29.100339 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.100310    5664 generic.go:356] "Generic (PLEG): container finished" podID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerID="0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd" exitCode=0
Oct 30 23:00:29.100464 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.100374    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"
Oct 30 23:00:29.100464 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.100389    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" event={"ID":"1a3fad52-b3fa-40e7-bac9-4d7e873f866f","Type":"ContainerDied","Data":"0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd"}
Oct 30 23:00:29.100552 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.100463    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57" event={"ID":"1a3fad52-b3fa-40e7-bac9-4d7e873f866f","Type":"ContainerDied","Data":"366413d5797ac74fb7ea4e472715b406d74eda58242b0b8ed81c9b58ef44d662"}
Oct 30 23:00:29.100552 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.100485    5664 scope.go:122] "RemoveContainer" containerID="0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd"
Oct 30 23:00:29.109309 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.109283    5664 scope.go:122] "RemoveContainer" containerID="fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982"
Oct 30 23:00:29.116735 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.116712    5664 scope.go:122] "RemoveContainer" containerID="0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd"
Oct 30 23:00:29.117018 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:00:29.116998    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd\": container with ID starting with 0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd not found: ID does not exist" containerID="0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd"
Oct 30 23:00:29.117085 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.117027    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd"} err="failed to get container status \"0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd\": rpc error: code = NotFound desc = could not find container \"0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd\": container with ID starting with 0edb5962b71d6a25d97e134b25bb604363844d317aa511b54c044e77e8e1dbcd not found: ID does not exist"
Oct 30 23:00:29.117085 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.117044    5664 scope.go:122] "RemoveContainer" containerID="fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982"
Oct 30 23:00:29.117308 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:00:29.117291    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982\": container with ID starting with fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982 not found: ID does not exist" containerID="fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982"
Oct 30 23:00:29.117359 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.117315    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982"} err="failed to get container status \"fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982\": rpc error: code = NotFound desc = could not find container \"fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982\": container with ID starting with fd30be73f4577aea1283e8a95cb7f5eb8343b5d9bbbee216549dc1e140957982 not found: ID does not exist"
Oct 30 23:00:29.124337 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.124312    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"]
Oct 30 23:00:29.129214 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.129194    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-5bc8cb7b8fxbr57"]
Oct 30 23:00:29.137398 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.137378    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:29.137483 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.137401    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rmxg\" (UniqueName: \"kubernetes.io/projected/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kube-api-access-4rmxg\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:29.137483 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.137411    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:29.137483 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.137421    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:29.137483 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.137429    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:29.137483 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.137437    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a3fad52-b3fa-40e7-bac9-4d7e873f866f-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:00:29.867024 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:00:29.866989    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" path="/var/lib/kubelet/pods/1a3fad52-b3fa-40e7-bac9-4d7e873f866f/volumes"
Oct 30 23:00:37.514181 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:00:37.514141    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:01:07.514989 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:01:07.514941    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:01:37.514821 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:01:37.514781    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:01:46.263415 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:01:46.263363    5664 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" podUID="320724d28f4bb4eb7790f5cce736ad83" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 23:02:07.515730 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:02:07.515690    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:02:37.558657 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:02:37.558616    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:03:07.514824 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:03:07.514786    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:03:18.358098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.358019    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"]
Oct 30 23:03:18.358489 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.358381    5664 cpu_manager.go:414] "containerMap: removing container" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main"
Oct 30 23:03:18.358489 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.358394    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main"
Oct 30 23:03:18.358489 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.358404    5664 cpu_manager.go:414] "containerMap: removing container" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="storage-initializer"
Oct 30 23:03:18.358489 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.358410    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="storage-initializer"
Oct 30 23:03:18.358722 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.358618    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a3fad52-b3fa-40e7-bac9-4d7e873f866f" containerName="main"
Oct 30 23:03:18.362135 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.362114    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.371074 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.371053    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"openshift-service-ca.crt\""
Oct 30 23:03:18.371328 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.371312    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"kube-root-ca.crt\""
Oct 30 23:03:18.371504 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.371487    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"default-dockercfg-ghpb2\""
Oct 30 23:03:18.371572 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.371511    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-7bc4bd10\"/\"llmisv20d1e8ef181cac7effb0fab7b8590b15-kserve-self-signed-certs\""
Oct 30 23:03:18.378150 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.378126    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"]
Oct 30 23:03:18.409201 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.409160    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tls-certs\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.409201 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.409203    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-dshm\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.409401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.409233    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-home\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.409401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.409352    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqwmr\" (UniqueName: \"kubernetes.io/projected/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kube-api-access-cqwmr\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.409512 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.409407    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.409512 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.409498    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.409602 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.409557    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-model-cache\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511017 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.510985    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tls-certs\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511017 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511020    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-dshm\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511270 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511047    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-home\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511270 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511076    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqwmr\" (UniqueName: \"kubernetes.io/projected/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kube-api-access-cqwmr\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511270 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511126    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511270 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511171    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511270 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511216    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-model-cache\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511528 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511469    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-home\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511647 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511620    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511971 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511672    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.511971 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.511708    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-model-cache\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.513566 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.513540    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-dshm\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.513685 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.513667    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tls-certs\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.524978 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.524956    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqwmr\" (UniqueName: \"kubernetes.io/projected/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kube-api-access-cqwmr\") pod \"llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") " pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.679956 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.679892    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:18.961923 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:18.961821    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"]
Oct 30 23:03:19.001979 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:19.001958    5664 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 23:03:19.834313 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:19.834273    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" event={"ID":"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89","Type":"ContainerStarted","Data":"d7a30717f849304c7ae43f4587d9524372dd24e273ea85d58cda39dcb1ac027e"}
Oct 30 23:03:24.858619 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:24.858519    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" event={"ID":"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89","Type":"ContainerStarted","Data":"55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e"}
Oct 30 23:03:25.864134 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:25.864104    5664 generic.go:356] "Generic (PLEG): container finished" podID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerID="55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e" exitCode=0
Oct 30 23:03:25.869128 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:25.869096    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" event={"ID":"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89","Type":"ContainerDied","Data":"55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e"}
Oct 30 23:03:26.870849 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:26.870758    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" event={"ID":"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89","Type":"ContainerStarted","Data":"a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387"}
Oct 30 23:03:26.870849 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:26.870813    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" event={"ID":"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89","Type":"ContainerStarted","Data":"2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa"}
Oct 30 23:03:26.900179 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:26.900122    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podStartSLOduration=2.1786098369999998 podStartE2EDuration="8.900103722s" totalImagesPullingTime="5.535228938s" totalInitContainerRuntime="1.186264947s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:03:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:03:26.897771543 +0000 UTC m=+2637.149235011" watchObservedRunningTime="2026-10-30 23:03:26.900103722 +0000 UTC m=+2637.151567189"
Oct 30 23:03:27.680933 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:27.680840    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:28.680546 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:28.680510    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:03:28.682036 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:28.681999    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:03:37.516218 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:03:37.516171    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:03:38.680825 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:38.680773    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:03:48.681062 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:48.681012    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:03:58.680849 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:03:58.680806    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:04:07.515851 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:04:07.515810    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:04:08.681037 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:04:08.680993    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:04:18.681411 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:04:18.681362    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:04:28.680972 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:04:28.680928    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:04:37.516575 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:04:37.516534    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:04:38.681176 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:04:38.681121    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:04:48.681285 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:04:48.681242    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:04:58.681265 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:04:58.681213    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:05:07.520142 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:05:07.520097    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:05:08.681039 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:08.680985    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:05:18.690914 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:18.690861    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:05:18.698708 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:18.698687    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:05:23.593427 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:23.593384    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"]
Oct 30 23:05:23.593942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:23.593786    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="modelcar" containerID="cri-o://a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387" gracePeriod=60
Oct 30 23:05:24.379461 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:24.379426    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-7bc4bd10_llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb_aef16ae6-2b30-42dc-ad47-8dbcbf49ca89/modelcar/0.log"
Oct 30 23:05:24.380102 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:24.380078    5664 generic.go:356] "Generic (PLEG): container finished" podID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerID="a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387" exitCode=143
Oct 30 23:05:24.380179 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:24.380151    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" event={"ID":"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89","Type":"ContainerDied","Data":"a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387"}
Oct 30 23:05:37.514477 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:05:37.514435    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:05:38.616638 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:38.616590    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" containerID="cri-o://2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa" gracePeriod=45
Oct 30 23:05:38.691873 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:38.691827    5664 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main" probeResult="failure" output="Get \"https://10.134.0.36:8000/health\": dial tcp 10.134.0.36:8000: connect: connection refused"
Oct 30 23:05:39.357512 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.357490    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-7bc4bd10_llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb_aef16ae6-2b30-42dc-ad47-8dbcbf49ca89/modelcar/0.log"
Oct 30 23:05:39.360204 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.360183    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:05:39.449423 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.449396    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-inference-service-7bc4bd10_llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb_aef16ae6-2b30-42dc-ad47-8dbcbf49ca89/modelcar/0.log"
Oct 30 23:05:39.450040 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.450019    5664 generic.go:356] "Generic (PLEG): container finished" podID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerID="2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa" exitCode=0
Oct 30 23:05:39.450126 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.450108    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"
Oct 30 23:05:39.450210 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.450108    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" event={"ID":"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89","Type":"ContainerDied","Data":"2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa"}
Oct 30 23:05:39.450292 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.450280    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb" event={"ID":"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89","Type":"ContainerDied","Data":"d7a30717f849304c7ae43f4587d9524372dd24e273ea85d58cda39dcb1ac027e"}
Oct 30 23:05:39.450330 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.450311    5664 scope.go:122] "RemoveContainer" containerID="a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387"
Oct 30 23:05:39.458729 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.458712    5664 scope.go:122] "RemoveContainer" containerID="2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa"
Oct 30 23:05:39.465799 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.465768    5664 scope.go:122] "RemoveContainer" containerID="55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e"
Oct 30 23:05:39.472739 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.472725    5664 scope.go:122] "RemoveContainer" containerID="a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387"
Oct 30 23:05:39.473008 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:05:39.472990    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387\": container with ID starting with a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387 not found: ID does not exist" containerID="a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387"
Oct 30 23:05:39.473064 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.473018    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387"} err="failed to get container status \"a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387\": rpc error: code = NotFound desc = could not find container \"a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387\": container with ID starting with a34f90931a8b92fa7da495de763f36e46133b01abc5539d66d7f4bd865893387 not found: ID does not exist"
Oct 30 23:05:39.473064 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.473034    5664 scope.go:122] "RemoveContainer" containerID="2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa"
Oct 30 23:05:39.473291 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:05:39.473270    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa\": container with ID starting with 2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa not found: ID does not exist" containerID="2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa"
Oct 30 23:05:39.473382 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.473296    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa"} err="failed to get container status \"2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa\": rpc error: code = NotFound desc = could not find container \"2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa\": container with ID starting with 2fa386bad80a07f01ab3dc8374412bec7d851e7806d3116f929047dac54123aa not found: ID does not exist"
Oct 30 23:05:39.473382 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.473311    5664 scope.go:122] "RemoveContainer" containerID="55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e"
Oct 30 23:05:39.473546 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:05:39.473528    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e\": container with ID starting with 55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e not found: ID does not exist" containerID="55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e"
Oct 30 23:05:39.473590 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.473550    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e"} err="failed to get container status \"55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e\": rpc error: code = NotFound desc = could not find container \"55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e\": container with ID starting with 55679c841c1d9e03e8eadb303b5cad7cc379fae7e727d5dae9c7f3b4f34bb58e not found: ID does not exist"
Oct 30 23:05:39.486804 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.486786    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-home\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-home\") pod \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") "
Oct 30 23:05:39.486887 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.486838    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kserve-provision-location\") pod \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") "
Oct 30 23:05:39.486887 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.486882    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tls-certs\" (UniqueName: \"kubernetes.io/secret/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tls-certs\") pod \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") "
Oct 30 23:05:39.486995 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.486928    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tmp-dir\") pod \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") "
Oct 30 23:05:39.486995 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.486947    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-model-cache\") pod \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") "
Oct 30 23:05:39.486995 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.486989    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kube-api-access-cqwmr\" (UniqueName: \"kubernetes.io/projected/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kube-api-access-cqwmr\") pod \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") "
Oct 30 23:05:39.487117 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.487044    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-dshm\") pod \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\" (UID: \"aef16ae6-2b30-42dc-ad47-8dbcbf49ca89\") "
Oct 30 23:05:39.487235 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.487212    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kserve-provision-location" pod "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" (UID: "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:05:39.487302 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.487223    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-model-cache" pod "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" (UID: "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:05:39.487416 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.487343    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:05:39.487416 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.487357    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:05:39.488091 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.487475    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-home" pod "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" (UID: "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:05:39.489440 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.489415    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kube-api-access-cqwmr" pod "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" (UID: "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89"). InnerVolumeSpecName "kube-api-access-cqwmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:05:39.489529 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.489463    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-dshm" pod "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" (UID: "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:05:39.489529 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.489462    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tls-certs" pod "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" (UID: "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:05:39.501592 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.501570    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tmp-dir" pod "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" (UID: "aef16ae6-2b30-42dc-ad47-8dbcbf49ca89"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:05:39.588511 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.588471    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:05:39.588511 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.588500    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:05:39.588511 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.588510    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:05:39.588511 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.588521    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqwmr\" (UniqueName: \"kubernetes.io/projected/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-kube-api-access-cqwmr\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:05:39.588778 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.588530    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:05:39.772841 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.772811    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"]
Oct 30 23:05:39.777168 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.777134    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-7bc4bd10/llmisvc-model-fb-opt-125m-oci-r-0fba19de-kserve-7687c48f9b25bfb"]
Oct 30 23:05:39.867609 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:05:39.867576    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" path="/var/lib/kubelet/pods/aef16ae6-2b30-42dc-ad47-8dbcbf49ca89/volumes"
Oct 30 23:06:07.514623 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:06:07.514559    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:06:37.515385 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:06:37.515343    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:07:07.515729 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:07:07.515685    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:07:37.514770 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:07:37.514733    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:08:07.515617 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:08:07.515529    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:08:27.950115 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.950084    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"]
Oct 30 23:08:27.950486 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.950385    5664 cpu_manager.go:414] "containerMap: removing container" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main"
Oct 30 23:08:27.950486 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.950397    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main"
Oct 30 23:08:27.950486 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.950416    5664 cpu_manager.go:414] "containerMap: removing container" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="modelcar"
Oct 30 23:08:27.950486 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.950421    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="modelcar"
Oct 30 23:08:27.950486 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.950439    5664 cpu_manager.go:414] "containerMap: removing container" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="modelcar-init"
Oct 30 23:08:27.950486 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.950443    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="modelcar-init"
Oct 30 23:08:27.950698 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.950490    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="modelcar"
Oct 30 23:08:27.950698 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.950500    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="aef16ae6-2b30-42dc-ad47-8dbcbf49ca89" containerName="main"
Oct 30 23:08:27.953731 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.953712    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:27.958996 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.958971    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"openshift-service-ca.crt\""
Oct 30 23:08:27.959735 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.959698    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"custom-route-timeout-test-kserve-self-signed-certs\""
Oct 30 23:08:27.959856 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.959798    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:08:27.959856 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.959832    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"seaweedfs-s3-creds\""
Oct 30 23:08:27.960032 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.959863    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"kube-root-ca.crt\""
Oct 30 23:08:27.960930 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.960889    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-caba0b4a\"/\"default-dockercfg-glldn\""
Oct 30 23:08:27.979924 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:27.979836    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"]
Oct 30 23:08:28.057251 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.057217    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f89hm\" (UniqueName: \"kubernetes.io/projected/d6202760-6146-4602-956c-ec302c247c22-kube-api-access-f89hm\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.057417 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.057259    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-tmp-dir\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.057458 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.057405    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6202760-6146-4602-956c-ec302c247c22-cabundle-cert\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.057458 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.057436    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-home\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.057458 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.057451    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-model-cache\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.057550 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.057466    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.057550 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.057531    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-dshm\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.057610 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.057564    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6202760-6146-4602-956c-ec302c247c22-tls-certs\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158276 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158241    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6202760-6146-4602-956c-ec302c247c22-tls-certs\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158466 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158285    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f89hm\" (UniqueName: \"kubernetes.io/projected/d6202760-6146-4602-956c-ec302c247c22-kube-api-access-f89hm\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158466 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158305    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-tmp-dir\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158596 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158487    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6202760-6146-4602-956c-ec302c247c22-cabundle-cert\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158596 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158536    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-home\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158596 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158571    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-model-cache\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158763 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158616    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158763 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158663    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-dshm\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158763 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158710    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-tmp-dir\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.158947 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.158925    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-home\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.159046 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.159029    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-kserve-provision-location\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.159262 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.159238    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6202760-6146-4602-956c-ec302c247c22-cabundle-cert\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.159373 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.159243    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-model-cache\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.161005 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.160989    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-dshm\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.161071 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.161040    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6202760-6146-4602-956c-ec302c247c22-tls-certs\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.175117 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.175094    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f89hm\" (UniqueName: \"kubernetes.io/projected/d6202760-6146-4602-956c-ec302c247c22-kube-api-access-f89hm\") pod \"custom-route-timeout-test-kserve-57d77d7f74-fs77t\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") " pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.268265 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.268170    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:28.555612 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:28.555501    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"]
Oct 30 23:08:28.559699 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:08:28.559661    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6202760_6146_4602_956c_ec302c247c22.slice/crio-e1557578d887ee12133450262a133123cdc22036dddd90e432d78cf9c13a8618 WatchSource:0}: Error finding container e1557578d887ee12133450262a133123cdc22036dddd90e432d78cf9c13a8618: Status 404 returned error can't find the container with id e1557578d887ee12133450262a133123cdc22036dddd90e432d78cf9c13a8618
Oct 30 23:08:29.124061 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:29.124019    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" event={"ID":"d6202760-6146-4602-956c-ec302c247c22","Type":"ContainerStarted","Data":"a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7"}
Oct 30 23:08:29.124061 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:29.124067    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" event={"ID":"d6202760-6146-4602-956c-ec302c247c22","Type":"ContainerStarted","Data":"e1557578d887ee12133450262a133123cdc22036dddd90e432d78cf9c13a8618"}
Oct 30 23:08:31.137607 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:31.137515    5664 generic.go:356] "Generic (PLEG): container finished" podID="d6202760-6146-4602-956c-ec302c247c22" containerID="a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7" exitCode=0
Oct 30 23:08:31.137607 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:31.137577    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" event={"ID":"d6202760-6146-4602-956c-ec302c247c22","Type":"ContainerDied","Data":"a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7"}
Oct 30 23:08:32.143496 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:32.143461    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" event={"ID":"d6202760-6146-4602-956c-ec302c247c22","Type":"ContainerStarted","Data":"26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f"}
Oct 30 23:08:32.175181 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:32.175110    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podStartSLOduration=2.91497851 podStartE2EDuration="5.175090689s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.260112179s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:08:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:08:32.172050627 +0000 UTC m=+2942.423514106" watchObservedRunningTime="2026-10-30 23:08:32.175090689 +0000 UTC m=+2942.426554158"
Oct 30 23:08:32.268636 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:32.268597    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:37.514559 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:08:37.514518    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:08:38.269346 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:38.269310    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:08:38.270734 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:38.270706    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:08:48.269117 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:48.269069    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:08:58.269238 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:08:58.269192    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:09:07.515641 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:09:07.515597    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:09:08.269229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:09:08.269189    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:09:18.268569 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:09:18.268477    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:09:28.269588 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:09:28.269541    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:09:37.515225 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:09:37.515179    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:09:38.268856 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:09:38.268810    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:09:48.268673 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:09:48.268628    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:09:58.268816 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:09:58.268775    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:10:07.516751 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:10:07.516704    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:10:08.268875 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:08.268834    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:10:18.269424 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:18.269382    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" probeResult="failure" output="Get \"https://10.134.0.37:8000/health\": dial tcp 10.134.0.37:8000: connect: connection refused"
Oct 30 23:10:28.279236 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:28.279199    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:10:28.287083 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:28.287053    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:10:33.497933 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:33.497885    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"]
Oct 30 23:10:37.514586 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:10:37.514542    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:10:48.525056 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:48.524941    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main" containerID="cri-o://26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f" gracePeriod=45
Oct 30 23:10:49.271448 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.271425    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:10:49.380545 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380467    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-home\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-home\") pod \"d6202760-6146-4602-956c-ec302c247c22\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") "
Oct 30 23:10:49.380545 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380511    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6202760-6146-4602-956c-ec302c247c22-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6202760-6146-4602-956c-ec302c247c22-tls-certs\") pod \"d6202760-6146-4602-956c-ec302c247c22\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") "
Oct 30 23:10:49.380852 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380555    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-model-cache\") pod \"d6202760-6146-4602-956c-ec302c247c22\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") "
Oct 30 23:10:49.380852 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380579    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-tmp-dir\") pod \"d6202760-6146-4602-956c-ec302c247c22\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") "
Oct 30 23:10:49.380852 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380596    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-dshm\") pod \"d6202760-6146-4602-956c-ec302c247c22\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") "
Oct 30 23:10:49.380852 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380622    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6202760-6146-4602-956c-ec302c247c22-kube-api-access-f89hm\" (UniqueName: \"kubernetes.io/projected/d6202760-6146-4602-956c-ec302c247c22-kube-api-access-f89hm\") pod \"d6202760-6146-4602-956c-ec302c247c22\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") "
Oct 30 23:10:49.380852 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380664    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6202760-6146-4602-956c-ec302c247c22-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6202760-6146-4602-956c-ec302c247c22-cabundle-cert\") pod \"d6202760-6146-4602-956c-ec302c247c22\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") "
Oct 30 23:10:49.380852 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380692    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-kserve-provision-location\") pod \"d6202760-6146-4602-956c-ec302c247c22\" (UID: \"d6202760-6146-4602-956c-ec302c247c22\") "
Oct 30 23:10:49.381200 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380837    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-model-cache" pod "d6202760-6146-4602-956c-ec302c247c22" (UID: "d6202760-6146-4602-956c-ec302c247c22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:10:49.381200 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.380986    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:10:49.381306 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.381222    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6202760-6146-4602-956c-ec302c247c22-cabundle-cert" pod "d6202760-6146-4602-956c-ec302c247c22" (UID: "d6202760-6146-4602-956c-ec302c247c22"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:10:49.381306 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.381238    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-home" pod "d6202760-6146-4602-956c-ec302c247c22" (UID: "d6202760-6146-4602-956c-ec302c247c22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:10:49.383239 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.383207    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6202760-6146-4602-956c-ec302c247c22-kube-api-access-f89hm" pod "d6202760-6146-4602-956c-ec302c247c22" (UID: "d6202760-6146-4602-956c-ec302c247c22"). InnerVolumeSpecName "kube-api-access-f89hm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:10:49.383339 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.383257    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6202760-6146-4602-956c-ec302c247c22-tls-certs" pod "d6202760-6146-4602-956c-ec302c247c22" (UID: "d6202760-6146-4602-956c-ec302c247c22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:10:49.383339 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.383295    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-dshm" pod "d6202760-6146-4602-956c-ec302c247c22" (UID: "d6202760-6146-4602-956c-ec302c247c22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:10:49.392683 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.392655    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-tmp-dir" pod "d6202760-6146-4602-956c-ec302c247c22" (UID: "d6202760-6146-4602-956c-ec302c247c22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:10:49.401809 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.401789    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-kserve-provision-location" pod "d6202760-6146-4602-956c-ec302c247c22" (UID: "d6202760-6146-4602-956c-ec302c247c22"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:10:49.482030 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.481999    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:10:49.482030 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.482025    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:10:49.482030 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.482035    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f89hm\" (UniqueName: \"kubernetes.io/projected/d6202760-6146-4602-956c-ec302c247c22-kube-api-access-f89hm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:10:49.482229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.482045    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6202760-6146-4602-956c-ec302c247c22-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:10:49.482229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.482055    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:10:49.482229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.482063    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6202760-6146-4602-956c-ec302c247c22-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:10:49.482229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.482071    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6202760-6146-4602-956c-ec302c247c22-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:10:49.744763 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.744726    5664 generic.go:356] "Generic (PLEG): container finished" podID="d6202760-6146-4602-956c-ec302c247c22" containerID="26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f" exitCode=0
Oct 30 23:10:49.745258 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.744786    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"
Oct 30 23:10:49.745258 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.744793    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" event={"ID":"d6202760-6146-4602-956c-ec302c247c22","Type":"ContainerDied","Data":"26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f"}
Oct 30 23:10:49.745258 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.744913    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t" event={"ID":"d6202760-6146-4602-956c-ec302c247c22","Type":"ContainerDied","Data":"e1557578d887ee12133450262a133123cdc22036dddd90e432d78cf9c13a8618"}
Oct 30 23:10:49.745258 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.744934    5664 scope.go:122] "RemoveContainer" containerID="26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f"
Oct 30 23:10:49.753815 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.753783    5664 scope.go:122] "RemoveContainer" containerID="a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7"
Oct 30 23:10:49.761097 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.761078    5664 scope.go:122] "RemoveContainer" containerID="26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f"
Oct 30 23:10:49.761349 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:10:49.761329    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f\": container with ID starting with 26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f not found: ID does not exist" containerID="26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f"
Oct 30 23:10:49.761403 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.761359    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f"} err="failed to get container status \"26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f\": rpc error: code = NotFound desc = could not find container \"26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f\": container with ID starting with 26cd5273f6e47f9bf8f7f89783a31442198077bc7880b6cc7db8b6ccb1a6ca2f not found: ID does not exist"
Oct 30 23:10:49.761403 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.761376    5664 scope.go:122] "RemoveContainer" containerID="a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7"
Oct 30 23:10:49.761611 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:10:49.761594    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7\": container with ID starting with a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7 not found: ID does not exist" containerID="a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7"
Oct 30 23:10:49.761654 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.761616    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7"} err="failed to get container status \"a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7\": rpc error: code = NotFound desc = could not find container \"a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7\": container with ID starting with a1fe23367e10c784fda49cb42cf81d1197f7b877dbe75aca0c064e85e5829ac7 not found: ID does not exist"
Oct 30 23:10:49.768113 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.768093    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"]
Oct 30 23:10:49.775937 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.775914    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-caba0b4a/custom-route-timeout-test-kserve-57d77d7f74-fs77t"]
Oct 30 23:10:49.867347 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:10:49.867313    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6202760-6146-4602-956c-ec302c247c22" path="/var/lib/kubelet/pods/d6202760-6146-4602-956c-ec302c247c22/volumes"
Oct 30 23:11:07.514930 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:11:07.514873    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:11:24.027575 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.027535    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"]
Oct 30 23:11:24.028172 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.028018    5664 cpu_manager.go:414] "containerMap: removing container" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="storage-initializer"
Oct 30 23:11:24.028172 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.028036    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="storage-initializer"
Oct 30 23:11:24.028172 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.028062    5664 cpu_manager.go:414] "containerMap: removing container" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main"
Oct 30 23:11:24.028172 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.028069    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main"
Oct 30 23:11:24.028172 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.028131    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6202760-6146-4602-956c-ec302c247c22" containerName="main"
Oct 30 23:11:24.031169 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.031147    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.036500 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.036468    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-gateway-1-openshift-default-dockercfg-j45tp\""
Oct 30 23:11:24.037793 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.037753    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-gw-ca-root-cert\""
Oct 30 23:11:24.037793 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.037755    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"openshift-service-ca.crt\""
Oct 30 23:11:24.037999 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.037837    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"kube-root-ca.crt\""
Oct 30 23:11:24.052466 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.052439    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"]
Oct 30 23:11:24.187367 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.187329    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd23ab10-040b-464e-91d0-3ac74366c3c5-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.187546 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.187371    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsq6b\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-kube-api-access-wsq6b\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.187546 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.187524    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-socket\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.187627 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.187555    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-envoy\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.187627 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.187579    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-certs\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.187744 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.187729    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-podinfo\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.187792 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.187759    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-credential-socket\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.187792 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.187782    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-data\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.187870 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.187803    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-token\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288230 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288154    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd23ab10-040b-464e-91d0-3ac74366c3c5-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288230 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288191    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsq6b\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-kube-api-access-wsq6b\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288230 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288231    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-socket\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288477 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288246    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-envoy\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288477 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288268    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-certs\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288477 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288311    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-podinfo\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288477 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288327    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-credential-socket\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288477 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288345    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-data\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288477 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288367    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-token\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288802 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288777    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-socket\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288864 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288817    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-certs\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288864 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288834    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-data\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.288992 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.288967    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd23ab10-040b-464e-91d0-3ac74366c3c5-istiod-ca-cert\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.289265 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.289207    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-credential-socket\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.290978 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.290957    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-envoy\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.291125 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.291107    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-podinfo\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.313201 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.313175    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-token\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.313522 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.313502    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsq6b\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-kube-api-access-wsq6b\") pod \"router-gateway-1-openshift-default-56d96d6b69-n8kqx\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") " pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.344594 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.344566    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:24.625024 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.624934    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"]
Oct 30 23:11:24.627974 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:11:24.627946    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd23ab10_040b_464e_91d0_3ac74366c3c5.slice/crio-9c54fc67afb00b84899669adc2e86dee6d4316d44bb8529ed4e287625b0e33be WatchSource:0}: Error finding container 9c54fc67afb00b84899669adc2e86dee6d4316d44bb8529ed4e287625b0e33be: Status 404 returned error can't find the container with id 9c54fc67afb00b84899669adc2e86dee6d4316d44bb8529ed4e287625b0e33be
Oct 30 23:11:24.630294 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.630263    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:11:24.630392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.630326    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:11:24.630392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.630352    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:11:24.889515 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.889412    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" event={"ID":"cd23ab10-040b-464e-91d0-3ac74366c3c5","Type":"ContainerStarted","Data":"7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47"}
Oct 30 23:11:24.889515 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.889465    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" event={"ID":"cd23ab10-040b-464e-91d0-3ac74366c3c5","Type":"ContainerStarted","Data":"9c54fc67afb00b84899669adc2e86dee6d4316d44bb8529ed4e287625b0e33be"}
Oct 30 23:11:24.951703 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:24.951640    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" podStartSLOduration=0.951626332 podStartE2EDuration="951.626332ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:11:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:11:24.951311378 +0000 UTC m=+3115.202774852" watchObservedRunningTime="2026-10-30 23:11:24.951626332 +0000 UTC m=+3115.203089873"
Oct 30 23:11:25.345616 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:25.345572    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:25.350685 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:25.350663    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:25.893734 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:25.893706    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:25.894600 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:25.894579    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:34.936880 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:34.936800    5664 generic.go:356] "Generic (PLEG): container finished" podID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerID="7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47" exitCode=0
Oct 30 23:11:34.936880 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:34.936850    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" event={"ID":"cd23ab10-040b-464e-91d0-3ac74366c3c5","Type":"ContainerDied","Data":"7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47"}
Oct 30 23:11:34.937394 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:34.937216    5664 scope.go:122] "RemoveContainer" containerID="7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47"
Oct 30 23:11:34.938235 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:34.938208    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:11:34.938291 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:34.938270    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:11:34.938324 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:34.938300    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:11:35.943170 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:35.943136    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" event={"ID":"cd23ab10-040b-464e-91d0-3ac74366c3c5","Type":"ContainerStarted","Data":"d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf"}
Oct 30 23:11:36.345607 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:36.345514    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:36.346485 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:36.346460    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 23:11:37.345159 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:37.345114    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 23:11:37.526291 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:11:37.526256    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:11:38.345869 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:38.345828    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 23:11:39.345858 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:39.345811    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 23:11:40.345573 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:40.345529    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 23:11:40.894051 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:40.894004    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:41.349096 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:41.349063    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:41.350160 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:41.350139    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:11:44.466383 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.466353    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"]
Oct 30 23:11:44.469808 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.469792    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.504758 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.504732    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"seaweedfs-s3-creds\""
Oct 30 23:11:44.504949 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.504844    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"router-with-refs-test-kserve-self-signed-certs\""
Oct 30 23:11:44.505775 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.505760    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:11:44.505840 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.505830    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-028f7809\"/\"default-dockercfg-7h88w\""
Oct 30 23:11:44.509069 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.509048    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"]
Oct 30 23:11:44.579917 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.579866    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9f666a3-78ea-4cd8-816e-911b37ce9aff-cabundle-cert\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.580079 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.579979    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-model-cache\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.580079 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.580000    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhlh7\" (UniqueName: \"kubernetes.io/projected/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kube-api-access-vhlh7\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.580079 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.580032    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-dshm\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.580179 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.580115    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-home\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.580179 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.580152    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tls-certs\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.580179 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.580170    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tmp-dir\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.580269 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.580202    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kserve-provision-location\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681364 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681330    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tls-certs\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681364 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681364    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tmp-dir\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681604 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681384    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kserve-provision-location\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681604 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681422    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9f666a3-78ea-4cd8-816e-911b37ce9aff-cabundle-cert\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681604 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681475    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-model-cache\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681604 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681501    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhlh7\" (UniqueName: \"kubernetes.io/projected/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kube-api-access-vhlh7\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681604 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681532    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-dshm\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681604 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681560    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-home\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681877 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681832    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tmp-dir\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.681977 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681939    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kserve-provision-location\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.682027 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681971    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-home\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.682027 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.681945    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-model-cache\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.682264 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.682245    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9f666a3-78ea-4cd8-816e-911b37ce9aff-cabundle-cert\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.684041 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.684022    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-dshm\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.684132 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.684116    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tls-certs\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.697264 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.697243    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhlh7\" (UniqueName: \"kubernetes.io/projected/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kube-api-access-vhlh7\") pod \"router-with-refs-test-kserve-569c6bbc86-8jh8j\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") " pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:44.784591 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:44.784503    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:45.083132 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:45.083098    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"]
Oct 30 23:11:45.086955 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:11:45.086920    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9f666a3_78ea_4cd8_816e_911b37ce9aff.slice/crio-3f496562040b52de9cbca735efe6f8036a0d132b7c177ffe1666ec42d9c4ade9 WatchSource:0}: Error finding container 3f496562040b52de9cbca735efe6f8036a0d132b7c177ffe1666ec42d9c4ade9: Status 404 returned error can't find the container with id 3f496562040b52de9cbca735efe6f8036a0d132b7c177ffe1666ec42d9c4ade9
Oct 30 23:11:45.983463 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:45.983419    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" event={"ID":"c9f666a3-78ea-4cd8-816e-911b37ce9aff","Type":"ContainerStarted","Data":"242e9960c85df8505da12d0e091e755d6e8ec06a39009f75af4ff10fa0ced7a2"}
Oct 30 23:11:45.983995 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:45.983486    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" event={"ID":"c9f666a3-78ea-4cd8-816e-911b37ce9aff","Type":"ContainerStarted","Data":"3f496562040b52de9cbca735efe6f8036a0d132b7c177ffe1666ec42d9c4ade9"}
Oct 30 23:11:46.263658 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:46.263559    5664 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" podUID="320724d28f4bb4eb7790f5cce736ad83" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 23:11:47.993703 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:47.993667    5664 generic.go:356] "Generic (PLEG): container finished" podID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerID="242e9960c85df8505da12d0e091e755d6e8ec06a39009f75af4ff10fa0ced7a2" exitCode=0
Oct 30 23:11:47.993703 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:47.993707    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" event={"ID":"c9f666a3-78ea-4cd8-816e-911b37ce9aff","Type":"ContainerDied","Data":"242e9960c85df8505da12d0e091e755d6e8ec06a39009f75af4ff10fa0ced7a2"}
Oct 30 23:11:49.001328 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:49.001292    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" event={"ID":"c9f666a3-78ea-4cd8-816e-911b37ce9aff","Type":"ContainerStarted","Data":"4cd4b7163b8fdeb5b71cb1441c8ee4403dccbeaa9a3cc4a788249e2dc225daaa"}
Oct 30 23:11:49.030718 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:49.030664    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podStartSLOduration=2.8590172640000002 podStartE2EDuration="5.030649912s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.171632648s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:11:49.028713228 +0000 UTC m=+3139.280176694" watchObservedRunningTime="2026-10-30 23:11:49.030649912 +0000 UTC m=+3139.282113379"
Oct 30 23:11:49.785676 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:49.785635    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:54.785402 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:54.785365    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:11:54.786939 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:11:54.786909    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:12:04.785406 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:12:04.785338    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:12:07.516670 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:12:07.516632    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:12:14.785542 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:12:14.785452    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:12:24.786022 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:12:24.785973    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:12:34.785967 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:12:34.785922    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:12:37.517345 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:12:37.517307    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:12:44.785227 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:12:44.785176    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:12:54.785059 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:12:54.785011    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:13:04.785653 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:13:04.785609    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:13:07.518602 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:13:07.518557    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:13:14.785457 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:13:14.785402    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:13:24.788494 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:13:24.788441    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:13:34.785472 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:13:34.785413    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" probeResult="failure" output="Get \"https://10.134.0.39:8000/health\": dial tcp 10.134.0.39:8000: connect: connection refused"
Oct 30 23:13:37.517808 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:13:37.517762    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:13:44.795168 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:13:44.795095    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:13:44.802744 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:13:44.802719    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:13:46.263428 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:13:46.263386    5664 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" podUID="320724d28f4bb4eb7790f5cce736ad83" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 23:13:49.968728 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:13:49.968691    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"]
Oct 30 23:14:04.993161 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:04.993124    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main" containerID="cri-o://4cd4b7163b8fdeb5b71cb1441c8ee4403dccbeaa9a3cc4a788249e2dc225daaa" gracePeriod=45
Oct 30 23:14:05.633235 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.633203    5664 generic.go:356] "Generic (PLEG): container finished" podID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerID="4cd4b7163b8fdeb5b71cb1441c8ee4403dccbeaa9a3cc4a788249e2dc225daaa" exitCode=0
Oct 30 23:14:05.633386 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.633286    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" event={"ID":"c9f666a3-78ea-4cd8-816e-911b37ce9aff","Type":"ContainerDied","Data":"4cd4b7163b8fdeb5b71cb1441c8ee4403dccbeaa9a3cc4a788249e2dc225daaa"}
Oct 30 23:14:05.741657 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.741626    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:14:05.845093 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845003    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-model-cache\") pod \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") "
Oct 30 23:14:05.845093 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845064    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9f666a3-78ea-4cd8-816e-911b37ce9aff-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9f666a3-78ea-4cd8-816e-911b37ce9aff-cabundle-cert\") pod \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") "
Oct 30 23:14:05.845093 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845083    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-dshm\") pod \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") "
Oct 30 23:14:05.845362 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845113    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kserve-provision-location\") pod \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") "
Oct 30 23:14:05.845362 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845141    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-home\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-home\") pod \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") "
Oct 30 23:14:05.845362 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845178    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tls-certs\") pod \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") "
Oct 30 23:14:05.845362 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845207    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tmp-dir\") pod \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") "
Oct 30 23:14:05.845362 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845266    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kube-api-access-vhlh7\" (UniqueName: \"kubernetes.io/projected/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kube-api-access-vhlh7\") pod \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\" (UID: \"c9f666a3-78ea-4cd8-816e-911b37ce9aff\") "
Oct 30 23:14:05.845362 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845337    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-model-cache" pod "c9f666a3-78ea-4cd8-816e-911b37ce9aff" (UID: "c9f666a3-78ea-4cd8-816e-911b37ce9aff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:05.845681 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845484    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9f666a3-78ea-4cd8-816e-911b37ce9aff-cabundle-cert" pod "c9f666a3-78ea-4cd8-816e-911b37ce9aff" (UID: "c9f666a3-78ea-4cd8-816e-911b37ce9aff"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:14:05.845681 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845574    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:05.845681 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.845596    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c9f666a3-78ea-4cd8-816e-911b37ce9aff-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:05.846269 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.846241    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-home" pod "c9f666a3-78ea-4cd8-816e-911b37ce9aff" (UID: "c9f666a3-78ea-4cd8-816e-911b37ce9aff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:05.847784 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.847749    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tls-certs" pod "c9f666a3-78ea-4cd8-816e-911b37ce9aff" (UID: "c9f666a3-78ea-4cd8-816e-911b37ce9aff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:14:05.847880 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.847792    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-dshm" pod "c9f666a3-78ea-4cd8-816e-911b37ce9aff" (UID: "c9f666a3-78ea-4cd8-816e-911b37ce9aff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:05.847976 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.847954    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kube-api-access-vhlh7" pod "c9f666a3-78ea-4cd8-816e-911b37ce9aff" (UID: "c9f666a3-78ea-4cd8-816e-911b37ce9aff"). InnerVolumeSpecName "kube-api-access-vhlh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:14:05.863822 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.863786    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tmp-dir" pod "c9f666a3-78ea-4cd8-816e-911b37ce9aff" (UID: "c9f666a3-78ea-4cd8-816e-911b37ce9aff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:05.868626 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.868601    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kserve-provision-location" pod "c9f666a3-78ea-4cd8-816e-911b37ce9aff" (UID: "c9f666a3-78ea-4cd8-816e-911b37ce9aff"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:05.946488 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.946461    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:05.946488 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.946487    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:05.946660 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.946498    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:05.946660 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.946509    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:05.946660 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.946518    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c9f666a3-78ea-4cd8-816e-911b37ce9aff-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:05.946660 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:05.946526    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhlh7\" (UniqueName: \"kubernetes.io/projected/c9f666a3-78ea-4cd8-816e-911b37ce9aff-kube-api-access-vhlh7\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:06.638971 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:06.638939    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"
Oct 30 23:14:06.639402 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:06.638941    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j" event={"ID":"c9f666a3-78ea-4cd8-816e-911b37ce9aff","Type":"ContainerDied","Data":"3f496562040b52de9cbca735efe6f8036a0d132b7c177ffe1666ec42d9c4ade9"}
Oct 30 23:14:06.639402 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:06.639083    5664 scope.go:122] "RemoveContainer" containerID="4cd4b7163b8fdeb5b71cb1441c8ee4403dccbeaa9a3cc4a788249e2dc225daaa"
Oct 30 23:14:06.648493 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:06.648477    5664 scope.go:122] "RemoveContainer" containerID="242e9960c85df8505da12d0e091e755d6e8ec06a39009f75af4ff10fa0ced7a2"
Oct 30 23:14:06.664011 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:06.663989    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"]
Oct 30 23:14:06.669315 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:06.669291    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-with-refs-test-kserve-569c6bbc86-8jh8j"]
Oct 30 23:14:07.515434 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:14:07.515396    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:14:07.867310 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:07.867223    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" path="/var/lib/kubelet/pods/c9f666a3-78ea-4cd8-816e-911b37ce9aff/volumes"
Oct 30 23:14:37.515232 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:14:37.515197    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:14:45.253112 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.253073    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"]
Oct 30 23:14:45.253604 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.253289    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy" containerID="cri-o://d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf" gracePeriod=30
Oct 30 23:14:45.716014 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.715976    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"]
Oct 30 23:14:45.716293 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.716280    5664 cpu_manager.go:414] "containerMap: removing container" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="storage-initializer"
Oct 30 23:14:45.716353 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.716294    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="storage-initializer"
Oct 30 23:14:45.716353 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.716308    5664 cpu_manager.go:414] "containerMap: removing container" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main"
Oct 30 23:14:45.716353 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.716313    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main"
Oct 30 23:14:45.716443 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.716377    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9f666a3-78ea-4cd8-816e-911b37ce9aff" containerName="main"
Oct 30 23:14:45.719670 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.719642    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.724742 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.724716    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:14:45.725192 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.725171    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"seaweedfs-s3-creds\""
Oct 30 23:14:45.725399 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.725369    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"default-dockercfg-pjngx\""
Oct 30 23:14:45.725487 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.725427    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Oct 30 23:14:45.725542 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.725526    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-dockercfg-mdl76\""
Oct 30 23:14:45.725659 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.725637    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"openshift-service-ca.crt\""
Oct 30 23:14:45.729206 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.729187    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"kube-root-ca.crt\""
Oct 30 23:14:45.739696 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.739664    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"]
Oct 30 23:14:45.802120 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.802084    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.802129    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.802203    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.802238    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.802276    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-home\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.802467 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.802389    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-dshm\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.802467 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.802424    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.802467 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.802446    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w957j\" (UniqueName: \"kubernetes.io/projected/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kube-api-access-w957j\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.903077 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.903032    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.903077 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.903075    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w957j\" (UniqueName: \"kubernetes.io/projected/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kube-api-access-w957j\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.903339 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.903143    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.903339 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.903185    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.903339 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.903216    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.903339 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.903250    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.903339 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.903306    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-home\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.903569 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.903359    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-dshm\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.904017 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.903934    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.904155 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.904024    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.904155 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.904064    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.906303 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.904450    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-home\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.906303 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.905518    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.906884 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.906857    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.907045 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.906927    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-dshm\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:45.920170 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:45.920132    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w957j\" (UniqueName: \"kubernetes.io/projected/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kube-api-access-w957j\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:46.036658 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:46.036568    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:46.325660 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:46.325573    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"]
Oct 30 23:14:46.328987 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:14:46.328951    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6880fcb_edb6_4f7c_aa36_c63f617f7f28.slice/crio-3d9d98e1ed1f6ec0b22252d14d8f8368ea1b3f8e3206ce65d3778549dcf7cde3 WatchSource:0}: Error finding container 3d9d98e1ed1f6ec0b22252d14d8f8368ea1b3f8e3206ce65d3778549dcf7cde3: Status 404 returned error can't find the container with id 3d9d98e1ed1f6ec0b22252d14d8f8368ea1b3f8e3206ce65d3778549dcf7cde3
Oct 30 23:14:46.811219 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:46.810613    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" event={"ID":"d6880fcb-edb6-4f7c-aa36-c63f617f7f28","Type":"ContainerStarted","Data":"41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7"}
Oct 30 23:14:46.811219 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:46.810687    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:46.811219 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:46.810718    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" event={"ID":"d6880fcb-edb6-4f7c-aa36-c63f617f7f28","Type":"ContainerStarted","Data":"3d9d98e1ed1f6ec0b22252d14d8f8368ea1b3f8e3206ce65d3778549dcf7cde3"}
Oct 30 23:14:47.817330 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:47.817288    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" event={"ID":"d6880fcb-edb6-4f7c-aa36-c63f617f7f28","Type":"ContainerStarted","Data":"af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90"}
Oct 30 23:14:49.827608 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:49.827576    5664 generic.go:356] "Generic (PLEG): container finished" podID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerID="af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90" exitCode=0
Oct 30 23:14:49.828119 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:49.827653    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" event={"ID":"d6880fcb-edb6-4f7c-aa36-c63f617f7f28","Type":"ContainerDied","Data":"af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90"}
Oct 30 23:14:50.517497 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.517424    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:14:50.649279 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649191    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-envoy\") pod \"cd23ab10-040b-464e-91d0-3ac74366c3c5\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") "
Oct 30 23:14:50.649279 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649232    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-data\") pod \"cd23ab10-040b-464e-91d0-3ac74366c3c5\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") "
Oct 30 23:14:50.649279 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649269    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-podinfo\") pod \"cd23ab10-040b-464e-91d0-3ac74366c3c5\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") "
Oct 30 23:14:50.649583 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649297    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-token\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-token\") pod \"cd23ab10-040b-464e-91d0-3ac74366c3c5\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") "
Oct 30 23:14:50.649583 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649533    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-data" pod "cd23ab10-040b-464e-91d0-3ac74366c3c5" (UID: "cd23ab10-040b-464e-91d0-3ac74366c3c5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:50.649738 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649716    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-kube-api-access-wsq6b\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-kube-api-access-wsq6b\") pod \"cd23ab10-040b-464e-91d0-3ac74366c3c5\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") "
Oct 30 23:14:50.649792 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649771    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-certs\") pod \"cd23ab10-040b-464e-91d0-3ac74366c3c5\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") "
Oct 30 23:14:50.649832 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649798    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-socket\") pod \"cd23ab10-040b-464e-91d0-3ac74366c3c5\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") "
Oct 30 23:14:50.649832 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649826    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-credential-socket\") pod \"cd23ab10-040b-464e-91d0-3ac74366c3c5\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") "
Oct 30 23:14:50.650422 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.649927    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd23ab10-040b-464e-91d0-3ac74366c3c5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd23ab10-040b-464e-91d0-3ac74366c3c5-istiod-ca-cert\") pod \"cd23ab10-040b-464e-91d0-3ac74366c3c5\" (UID: \"cd23ab10-040b-464e-91d0-3ac74366c3c5\") "
Oct 30 23:14:50.650422 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.650091    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-certs" pod "cd23ab10-040b-464e-91d0-3ac74366c3c5" (UID: "cd23ab10-040b-464e-91d0-3ac74366c3c5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:50.650422 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.650268    5664 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-data\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:50.650422 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.650293    5664 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:50.650422 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.650260    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-credential-socket" pod "cd23ab10-040b-464e-91d0-3ac74366c3c5" (UID: "cd23ab10-040b-464e-91d0-3ac74366c3c5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:50.650422 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.650307    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-socket" pod "cd23ab10-040b-464e-91d0-3ac74366c3c5" (UID: "cd23ab10-040b-464e-91d0-3ac74366c3c5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:50.650808 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.650577    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd23ab10-040b-464e-91d0-3ac74366c3c5-istiod-ca-cert" pod "cd23ab10-040b-464e-91d0-3ac74366c3c5" (UID: "cd23ab10-040b-464e-91d0-3ac74366c3c5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:14:50.652494 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.652463    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-token" pod "cd23ab10-040b-464e-91d0-3ac74366c3c5" (UID: "cd23ab10-040b-464e-91d0-3ac74366c3c5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:14:50.652716 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.652697    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-envoy" pod "cd23ab10-040b-464e-91d0-3ac74366c3c5" (UID: "cd23ab10-040b-464e-91d0-3ac74366c3c5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:14:50.653176 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.653157    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-kube-api-access-wsq6b" pod "cd23ab10-040b-464e-91d0-3ac74366c3c5" (UID: "cd23ab10-040b-464e-91d0-3ac74366c3c5"). InnerVolumeSpecName "kube-api-access-wsq6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:14:50.653263 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.653242    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-podinfo" pod "cd23ab10-040b-464e-91d0-3ac74366c3c5" (UID: "cd23ab10-040b-464e-91d0-3ac74366c3c5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:14:50.751043 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.750991    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsq6b\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-kube-api-access-wsq6b\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:50.751043 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.751025    5664 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-workload-socket\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:50.751043 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.751035    5664 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-credential-socket\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:50.751043 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.751046    5664 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd23ab10-040b-464e-91d0-3ac74366c3c5-istiod-ca-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:50.751043 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.751055    5664 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-envoy\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:50.751467 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.751063    5664 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-podinfo\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:50.751467 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.751072    5664 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd23ab10-040b-464e-91d0-3ac74366c3c5-istio-token\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:14:50.835000 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.834881    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" event={"ID":"d6880fcb-edb6-4f7c-aa36-c63f617f7f28","Type":"ContainerStarted","Data":"ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6"}
Oct 30 23:14:50.836666 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.836635    5664 generic.go:356] "Generic (PLEG): container finished" podID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerID="d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf" exitCode=0
Oct 30 23:14:50.836789 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.836684    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"
Oct 30 23:14:50.836789 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.836715    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" event={"ID":"cd23ab10-040b-464e-91d0-3ac74366c3c5","Type":"ContainerDied","Data":"d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf"}
Oct 30 23:14:50.836789 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.836770    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx" event={"ID":"cd23ab10-040b-464e-91d0-3ac74366c3c5","Type":"ContainerDied","Data":"9c54fc67afb00b84899669adc2e86dee6d4316d44bb8529ed4e287625b0e33be"}
Oct 30 23:14:50.836912 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.836794    5664 scope.go:122] "RemoveContainer" containerID="d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf"
Oct 30 23:14:50.846824 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.846799    5664 scope.go:122] "RemoveContainer" containerID="7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47"
Oct 30 23:14:50.855321 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.855296    5664 scope.go:122] "RemoveContainer" containerID="d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf"
Oct 30 23:14:50.855596 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:14:50.855572    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf\": container with ID starting with d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf not found: ID does not exist" containerID="d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf"
Oct 30 23:14:50.855668 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.855605    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf"} err="failed to get container status \"d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf\": rpc error: code = NotFound desc = could not find container \"d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf\": container with ID starting with d19e6ca0119a7f5b5864702386daf71616cfbc108ff767275714c7a0833dadbf not found: ID does not exist"
Oct 30 23:14:50.855668 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.855632    5664 scope.go:122] "RemoveContainer" containerID="7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47"
Oct 30 23:14:50.855984 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:14:50.855959    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47\": container with ID starting with 7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47 not found: ID does not exist" containerID="7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47"
Oct 30 23:14:50.856075 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.855994    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47"} err="failed to get container status \"7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47\": rpc error: code = NotFound desc = could not find container \"7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47\": container with ID starting with 7150cada7394c5fe82de0c9e49253cf90ab61d304ea8f0c8e7cc89446faa3b47 not found: ID does not exist"
Oct 30 23:14:50.869702 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.869646    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podStartSLOduration=3.567390159 podStartE2EDuration="5.869631094s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.302240935s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:14:50.86778163 +0000 UTC m=+3321.119245099" watchObservedRunningTime="2026-10-30 23:14:50.869631094 +0000 UTC m=+3321.121094591"
Oct 30 23:14:50.881913 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.881869    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"]
Oct 30 23:14:50.889064 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:50.889030    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-028f7809/router-gateway-1-openshift-default-56d96d6b69-n8kqx"]
Oct 30 23:14:51.873806 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:51.873759    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" path="/var/lib/kubelet/pods/cd23ab10-040b-464e-91d0-3ac74366c3c5/volumes"
Oct 30 23:14:56.037300 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:56.037259    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:56.037757 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:56.037334    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:14:56.038645 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:56.038604    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:14:56.051294 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:14:56.051270    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:15:06.038041 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:15:06.037997    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:15:07.515799 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:15:07.515755    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:15:16.037936 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:15:16.037820    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:15:26.037163 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:15:26.037110    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:15:36.037213 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:15:36.037162    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:15:37.516176 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:15:37.516132    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:15:46.037583 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:15:46.037524    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:15:56.037050 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:15:56.037004    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:16:06.037392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:16:06.037339    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:16:07.515518 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:16:07.515476    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:16:16.037139 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:16:16.037027    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:16:26.037564 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:16:26.037505    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:16:36.037818 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:16:36.037766    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" probeResult="failure" output="Get \"https://10.134.0.40:8001/health\": dial tcp 10.134.0.40:8001: connect: connection refused"
Oct 30 23:16:37.524424 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:16:37.524379    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:16:46.046280 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:16:46.046206    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:16:46.057946 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:16:46.057919    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:16:54.983024 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:16:54.982993    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"]
Oct 30 23:17:07.514657 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:17:07.514618    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:17:10.011563 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:10.011526    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main" containerID="cri-o://ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6" gracePeriod=45
Oct 30 23:17:10.628700 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:10.628669    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="llm-d-routing-sidecar" containerID="cri-o://41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7" gracePeriod=45
Oct 30 23:17:10.874413 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:10.874386    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:17:11.007812 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.007773    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-model-cache\") pod \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") "
Oct 30 23:17:11.007812 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.007811    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-dshm\") pod \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") "
Oct 30 23:17:11.008098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.007849    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tmp-dir\") pod \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") "
Oct 30 23:17:11.008098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.007891    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-cabundle-cert\") pod \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") "
Oct 30 23:17:11.008098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.007946    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kserve-provision-location\") pod \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") "
Oct 30 23:17:11.008098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.007974    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kube-api-access-w957j\" (UniqueName: \"kubernetes.io/projected/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kube-api-access-w957j\") pod \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") "
Oct 30 23:17:11.008098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.008042    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-home\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-home\") pod \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") "
Oct 30 23:17:11.008098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.008072    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tls-certs\" (UniqueName: \"kubernetes.io/secret/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tls-certs\") pod \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\" (UID: \"d6880fcb-edb6-4f7c-aa36-c63f617f7f28\") "
Oct 30 23:17:11.008395 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.008202    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-model-cache" pod "d6880fcb-edb6-4f7c-aa36-c63f617f7f28" (UID: "d6880fcb-edb6-4f7c-aa36-c63f617f7f28"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:17:11.008395 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.008288    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-cabundle-cert" pod "d6880fcb-edb6-4f7c-aa36-c63f617f7f28" (UID: "d6880fcb-edb6-4f7c-aa36-c63f617f7f28"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:17:11.008395 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.008357    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:11.008863 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.008789    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-home" pod "d6880fcb-edb6-4f7c-aa36-c63f617f7f28" (UID: "d6880fcb-edb6-4f7c-aa36-c63f617f7f28"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:17:11.010520 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.010492    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tls-certs" pod "d6880fcb-edb6-4f7c-aa36-c63f617f7f28" (UID: "d6880fcb-edb6-4f7c-aa36-c63f617f7f28"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:17:11.010753 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.010727    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-dshm" pod "d6880fcb-edb6-4f7c-aa36-c63f617f7f28" (UID: "d6880fcb-edb6-4f7c-aa36-c63f617f7f28"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:17:11.010846 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.010778    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kube-api-access-w957j" pod "d6880fcb-edb6-4f7c-aa36-c63f617f7f28" (UID: "d6880fcb-edb6-4f7c-aa36-c63f617f7f28"). InnerVolumeSpecName "kube-api-access-w957j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:17:11.024669 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.024643    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tmp-dir" pod "d6880fcb-edb6-4f7c-aa36-c63f617f7f28" (UID: "d6880fcb-edb6-4f7c-aa36-c63f617f7f28"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:17:11.035825 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.035799    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kserve-provision-location" pod "d6880fcb-edb6-4f7c-aa36-c63f617f7f28" (UID: "d6880fcb-edb6-4f7c-aa36-c63f617f7f28"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:17:11.109392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.109309    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:11.109392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.109339    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:11.109392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.109351    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w957j\" (UniqueName: \"kubernetes.io/projected/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-kube-api-access-w957j\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:11.109392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.109361    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:11.109392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.109369    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:11.109392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.109377    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:11.109392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.109384    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6880fcb-edb6-4f7c-aa36-c63f617f7f28-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:17:11.452295 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.452257    5664 generic.go:356] "Generic (PLEG): container finished" podID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerID="ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6" exitCode=0
Oct 30 23:17:11.452295 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.452291    5664 generic.go:356] "Generic (PLEG): container finished" podID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerID="41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7" exitCode=0
Oct 30 23:17:11.452502 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.452313    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" event={"ID":"d6880fcb-edb6-4f7c-aa36-c63f617f7f28","Type":"ContainerDied","Data":"ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6"}
Oct 30 23:17:11.452502 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.452324    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"
Oct 30 23:17:11.452502 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.452368    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" event={"ID":"d6880fcb-edb6-4f7c-aa36-c63f617f7f28","Type":"ContainerDied","Data":"41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7"}
Oct 30 23:17:11.452502 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.452394    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc" event={"ID":"d6880fcb-edb6-4f7c-aa36-c63f617f7f28","Type":"ContainerDied","Data":"3d9d98e1ed1f6ec0b22252d14d8f8368ea1b3f8e3206ce65d3778549dcf7cde3"}
Oct 30 23:17:11.452502 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.452421    5664 scope.go:122] "RemoveContainer" containerID="ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6"
Oct 30 23:17:11.465318 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.465289    5664 scope.go:122] "RemoveContainer" containerID="af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90"
Oct 30 23:17:11.473075 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.473049    5664 scope.go:122] "RemoveContainer" containerID="41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7"
Oct 30 23:17:11.480441 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.480417    5664 scope.go:122] "RemoveContainer" containerID="ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6"
Oct 30 23:17:11.480800 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:17:11.480778    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6\": container with ID starting with ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6 not found: ID does not exist" containerID="ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6"
Oct 30 23:17:11.480915 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.480810    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6"} err="failed to get container status \"ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6\": rpc error: code = NotFound desc = could not find container \"ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6\": container with ID starting with ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6 not found: ID does not exist"
Oct 30 23:17:11.480915 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.480830    5664 scope.go:122] "RemoveContainer" containerID="af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90"
Oct 30 23:17:11.481387 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:17:11.481364    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90\": container with ID starting with af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90 not found: ID does not exist" containerID="af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90"
Oct 30 23:17:11.481451 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.481393    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90"} err="failed to get container status \"af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90\": rpc error: code = NotFound desc = could not find container \"af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90\": container with ID starting with af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90 not found: ID does not exist"
Oct 30 23:17:11.481451 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.481408    5664 scope.go:122] "RemoveContainer" containerID="41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7"
Oct 30 23:17:11.481689 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:17:11.481665    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7\": container with ID starting with 41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7 not found: ID does not exist" containerID="41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7"
Oct 30 23:17:11.481740 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.481699    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7"} err="failed to get container status \"41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7\": rpc error: code = NotFound desc = could not find container \"41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7\": container with ID starting with 41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7 not found: ID does not exist"
Oct 30 23:17:11.481740 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.481720    5664 scope.go:122] "RemoveContainer" containerID="ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6"
Oct 30 23:17:11.482051 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.482013    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6"} err="failed to get container status \"ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6\": rpc error: code = NotFound desc = could not find container \"ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6\": container with ID starting with ce88ce7383077684936dfce6e766ea95ade1c6768e4b20bee19e2ec5c44ce7a6 not found: ID does not exist"
Oct 30 23:17:11.482051 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.482050    5664 scope.go:122] "RemoveContainer" containerID="af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90"
Oct 30 23:17:11.482395 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.482361    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90"} err="failed to get container status \"af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90\": rpc error: code = NotFound desc = could not find container \"af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90\": container with ID starting with af3d742873299e9469b97b27941f90c28ad744379d984fc339cf715cec556b90 not found: ID does not exist"
Oct 30 23:17:11.482395 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.482387    5664 scope.go:122] "RemoveContainer" containerID="41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7"
Oct 30 23:17:11.482862 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.482831    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7"} err="failed to get container status \"41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7\": rpc error: code = NotFound desc = could not find container \"41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7\": container with ID starting with 41ba5d72d1fbffa41f30c322184f8f1ade5b75bed2323f035dd63d3d196890e7 not found: ID does not exist"
Oct 30 23:17:11.483100 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.483083    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"]
Oct 30 23:17:11.487052 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.487031    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-5d887b9874z84hc"]
Oct 30 23:17:11.868589 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:11.868510    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" path="/var/lib/kubelet/pods/d6880fcb-edb6-4f7c-aa36-c63f617f7f28/volumes"
Oct 30 23:17:37.515546 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:17:37.515508    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:17:47.447234 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447200    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"]
Oct 30 23:17:47.447725 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447682    5664 cpu_manager.go:414] "containerMap: removing container" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy"
Oct 30 23:17:47.447725 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447711    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy"
Oct 30 23:17:47.447842 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447736    5664 cpu_manager.go:414] "containerMap: removing container" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="storage-initializer"
Oct 30 23:17:47.447842 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447743    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="storage-initializer"
Oct 30 23:17:47.447842 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447755    5664 cpu_manager.go:414] "containerMap: removing container" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="llm-d-routing-sidecar"
Oct 30 23:17:47.447842 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447762    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="llm-d-routing-sidecar"
Oct 30 23:17:47.447842 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447778    5664 cpu_manager.go:414] "containerMap: removing container" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main"
Oct 30 23:17:47.447842 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447785    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main"
Oct 30 23:17:47.448173 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447854    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="main"
Oct 30 23:17:47.448173 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447873    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy"
Oct 30 23:17:47.448173 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447885    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6880fcb-edb6-4f7c-aa36-c63f617f7f28" containerName="llm-d-routing-sidecar"
Oct 30 23:17:47.448173 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447983    5664 cpu_manager.go:414] "containerMap: removing container" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy"
Oct 30 23:17:47.448173 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.447994    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy"
Oct 30 23:17:47.448173 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.448090    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd23ab10-040b-464e-91d0-3ac74366c3c5" containerName="istio-proxy"
Oct 30 23:17:47.451338 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.451316    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.467444 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.467418    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"seaweedfs-s3-creds\""
Oct 30 23:17:47.467590 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.467460    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-qbdtm\""
Oct 30 23:17:47.467590 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.467540    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Oct 30 23:17:47.467590 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.467569    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-dockercfg-8blpt\""
Oct 30 23:17:47.471172 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.471151    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Oct 30 23:17:47.471292 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.471278    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:17:47.472126 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.472110    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Oct 30 23:17:47.484167 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.484136    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"]
Oct 30 23:17:47.509569 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.509538    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.509569 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.509569    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-home\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.509735 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.509587    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.509735 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.509607    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-model-cache\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.509735 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.509655    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tjs8\" (UniqueName: \"kubernetes.io/projected/efa1fd48-214b-4cbc-aa32-67700fee6163-kube-api-access-5tjs8\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.509835 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.509745    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-dshm\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.509835 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.509762    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa1fd48-214b-4cbc-aa32-67700fee6163-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.509910 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.509832    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa1fd48-214b-4cbc-aa32-67700fee6163-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.610855 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.610822    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.610855 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.610857    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-home\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611158 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.610884    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611158 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.610927    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-model-cache\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611158 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.610952    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tjs8\" (UniqueName: \"kubernetes.io/projected/efa1fd48-214b-4cbc-aa32-67700fee6163-kube-api-access-5tjs8\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611158 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.610992    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-dshm\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611158 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.611018    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa1fd48-214b-4cbc-aa32-67700fee6163-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611158 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.611075    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa1fd48-214b-4cbc-aa32-67700fee6163-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611464 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.611305    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611464 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.611366    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-model-cache\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611620 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.611601    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-home\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611731 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.611705    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.611893 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.611873    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa1fd48-214b-4cbc-aa32-67700fee6163-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.613729 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.613711    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa1fd48-214b-4cbc-aa32-67700fee6163-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.613842 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.613802    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-dshm\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.627477 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.627448    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tjs8\" (UniqueName: \"kubernetes.io/projected/efa1fd48-214b-4cbc-aa32-67700fee6163-kube-api-access-5tjs8\") pod \"custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:47.769645 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:47.769547    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:48.064860 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:48.064651    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"]
Oct 30 23:17:48.067566 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:17:48.067539    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa1fd48_214b_4cbc_aa32_67700fee6163.slice/crio-7270c98435a693485e5d60ed7a28f775200d5768104aed6d6e822ec0dd9fe67c WatchSource:0}: Error finding container 7270c98435a693485e5d60ed7a28f775200d5768104aed6d6e822ec0dd9fe67c: Status 404 returned error can't find the container with id 7270c98435a693485e5d60ed7a28f775200d5768104aed6d6e822ec0dd9fe67c
Oct 30 23:17:48.599978 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:48.599937    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" event={"ID":"efa1fd48-214b-4cbc-aa32-67700fee6163","Type":"ContainerStarted","Data":"2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5"}
Oct 30 23:17:48.600418 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:48.599990    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" event={"ID":"efa1fd48-214b-4cbc-aa32-67700fee6163","Type":"ContainerStarted","Data":"7270c98435a693485e5d60ed7a28f775200d5768104aed6d6e822ec0dd9fe67c"}
Oct 30 23:17:48.600418 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:48.600044    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:49.605749 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:49.605710    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" event={"ID":"efa1fd48-214b-4cbc-aa32-67700fee6163","Type":"ContainerStarted","Data":"f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1"}
Oct 30 23:17:51.618975 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:51.618929    5664 generic.go:356] "Generic (PLEG): container finished" podID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerID="f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1" exitCode=0
Oct 30 23:17:51.619369 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:51.619002    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" event={"ID":"efa1fd48-214b-4cbc-aa32-67700fee6163","Type":"ContainerDied","Data":"f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1"}
Oct 30 23:17:52.625951 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:52.625915    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" event={"ID":"efa1fd48-214b-4cbc-aa32-67700fee6163","Type":"ContainerStarted","Data":"b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387"}
Oct 30 23:17:52.657887 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:52.657823    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podStartSLOduration=3.390397954 podStartE2EDuration="5.657806745s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.267408791s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:17:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:17:52.655929879 +0000 UTC m=+3502.907393346" watchObservedRunningTime="2026-10-30 23:17:52.657806745 +0000 UTC m=+3502.909270211"
Oct 30 23:17:57.770554 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:57.770515    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:57.771041 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:57.770855    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:17:57.771925 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:57.771877    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:17:58.664153 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:17:58.664125    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:18:07.514958 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:18:07.514917    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:18:07.770797 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:18:07.770691    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:18:17.770646 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:18:17.770595    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:18:27.770801 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:18:27.770753    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:18:37.515349 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:18:37.515303    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:18:37.770262 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:18:37.770161    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:18:47.772732 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:18:47.772683    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:18:57.771098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:18:57.771043    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:19:07.515987 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:19:07.515873    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:19:07.770882 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:19:07.770777    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:19:17.770544 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:19:17.770477    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:19:27.771056 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:19:27.771003    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:19:37.533241 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:19:37.533201    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:19:37.770382 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:19:37.770334    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" probeResult="failure" output="Get \"https://10.134.0.41:8001/health\": dial tcp 10.134.0.41:8001: connect: connection refused"
Oct 30 23:19:47.785337 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:19:47.785257    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:19:47.797005 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:19:47.796978    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:19:56.511060 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:19:56.509764    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"]
Oct 30 23:20:07.514998 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:20:07.514946    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:20:11.540257 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:11.540216    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main" containerID="cri-o://b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387" gracePeriod=45
Oct 30 23:20:12.158065 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.158012    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="llm-d-routing-sidecar" containerID="cri-o://2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5" gracePeriod=45
Oct 30 23:20:12.228439 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.228410    5664 generic.go:356] "Generic (PLEG): container finished" podID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerID="b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387" exitCode=0
Oct 30 23:20:12.228569 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.228452    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" event={"ID":"efa1fd48-214b-4cbc-aa32-67700fee6163","Type":"ContainerDied","Data":"b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387"}
Oct 30 23:20:12.401491 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.401464    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:20:12.575584 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.575480    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/efa1fd48-214b-4cbc-aa32-67700fee6163-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa1fd48-214b-4cbc-aa32-67700fee6163-cabundle-cert\") pod \"efa1fd48-214b-4cbc-aa32-67700fee6163\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") "
Oct 30 23:20:12.575584 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.575540    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/efa1fd48-214b-4cbc-aa32-67700fee6163-kube-api-access-5tjs8\" (UniqueName: \"kubernetes.io/projected/efa1fd48-214b-4cbc-aa32-67700fee6163-kube-api-access-5tjs8\") pod \"efa1fd48-214b-4cbc-aa32-67700fee6163\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") "
Oct 30 23:20:12.575584 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.575565    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/efa1fd48-214b-4cbc-aa32-67700fee6163-tls-certs\" (UniqueName: \"kubernetes.io/secret/efa1fd48-214b-4cbc-aa32-67700fee6163-tls-certs\") pod \"efa1fd48-214b-4cbc-aa32-67700fee6163\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") "
Oct 30 23:20:12.576241 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.575681    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-tmp-dir\") pod \"efa1fd48-214b-4cbc-aa32-67700fee6163\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") "
Oct 30 23:20:12.576241 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.575782    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-dshm\") pod \"efa1fd48-214b-4cbc-aa32-67700fee6163\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") "
Oct 30 23:20:12.576241 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.575826    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-kserve-provision-location\") pod \"efa1fd48-214b-4cbc-aa32-67700fee6163\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") "
Oct 30 23:20:12.576241 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.575866    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-home\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-home\") pod \"efa1fd48-214b-4cbc-aa32-67700fee6163\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") "
Oct 30 23:20:12.576241 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.575891    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-model-cache\") pod \"efa1fd48-214b-4cbc-aa32-67700fee6163\" (UID: \"efa1fd48-214b-4cbc-aa32-67700fee6163\") "
Oct 30 23:20:12.576241 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.576025    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa1fd48-214b-4cbc-aa32-67700fee6163-cabundle-cert" pod "efa1fd48-214b-4cbc-aa32-67700fee6163" (UID: "efa1fd48-214b-4cbc-aa32-67700fee6163"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:20:12.576241 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.576237    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/efa1fd48-214b-4cbc-aa32-67700fee6163-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:12.576702 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.576644    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-model-cache" pod "efa1fd48-214b-4cbc-aa32-67700fee6163" (UID: "efa1fd48-214b-4cbc-aa32-67700fee6163"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:20:12.576702 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.576671    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-home" pod "efa1fd48-214b-4cbc-aa32-67700fee6163" (UID: "efa1fd48-214b-4cbc-aa32-67700fee6163"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:20:12.578153 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.578124    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa1fd48-214b-4cbc-aa32-67700fee6163-kube-api-access-5tjs8" pod "efa1fd48-214b-4cbc-aa32-67700fee6163" (UID: "efa1fd48-214b-4cbc-aa32-67700fee6163"). InnerVolumeSpecName "kube-api-access-5tjs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:20:12.578378 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.578354    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa1fd48-214b-4cbc-aa32-67700fee6163-tls-certs" pod "efa1fd48-214b-4cbc-aa32-67700fee6163" (UID: "efa1fd48-214b-4cbc-aa32-67700fee6163"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:20:12.578457 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.578443    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-dshm" pod "efa1fd48-214b-4cbc-aa32-67700fee6163" (UID: "efa1fd48-214b-4cbc-aa32-67700fee6163"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:20:12.593568 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.593540    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-tmp-dir" pod "efa1fd48-214b-4cbc-aa32-67700fee6163" (UID: "efa1fd48-214b-4cbc-aa32-67700fee6163"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:20:12.605276 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.605249    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-kserve-provision-location" pod "efa1fd48-214b-4cbc-aa32-67700fee6163" (UID: "efa1fd48-214b-4cbc-aa32-67700fee6163"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:20:12.676844 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.676808    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:12.676844 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.676838    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:12.676844 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.676847    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:12.676844 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.676856    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:12.677105 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.676870    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efa1fd48-214b-4cbc-aa32-67700fee6163-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:12.677105 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.676880    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tjs8\" (UniqueName: \"kubernetes.io/projected/efa1fd48-214b-4cbc-aa32-67700fee6163-kube-api-access-5tjs8\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:12.677105 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:12.676888    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efa1fd48-214b-4cbc-aa32-67700fee6163-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:20:13.233915 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.233861    5664 generic.go:356] "Generic (PLEG): container finished" podID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerID="2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5" exitCode=0
Oct 30 23:20:13.234162 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.233946    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"
Oct 30 23:20:13.234162 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.233955    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" event={"ID":"efa1fd48-214b-4cbc-aa32-67700fee6163","Type":"ContainerDied","Data":"2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5"}
Oct 30 23:20:13.234162 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.234017    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2" event={"ID":"efa1fd48-214b-4cbc-aa32-67700fee6163","Type":"ContainerDied","Data":"7270c98435a693485e5d60ed7a28f775200d5768104aed6d6e822ec0dd9fe67c"}
Oct 30 23:20:13.234162 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.234048    5664 scope.go:122] "RemoveContainer" containerID="b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387"
Oct 30 23:20:13.243304 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.243281    5664 scope.go:122] "RemoveContainer" containerID="f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1"
Oct 30 23:20:13.250640 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.250612    5664 scope.go:122] "RemoveContainer" containerID="2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5"
Oct 30 23:20:13.259820 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.259796    5664 scope.go:122] "RemoveContainer" containerID="b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387"
Oct 30 23:20:13.260191 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:20:13.260173    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387\": container with ID starting with b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387 not found: ID does not exist" containerID="b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387"
Oct 30 23:20:13.260243 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.260202    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387"} err="failed to get container status \"b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387\": rpc error: code = NotFound desc = could not find container \"b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387\": container with ID starting with b75e1e5b1e47636696958406b67a4ea48e521f72acda50c78d9d9ba4956b4387 not found: ID does not exist"
Oct 30 23:20:13.260243 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.260218    5664 scope.go:122] "RemoveContainer" containerID="f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1"
Oct 30 23:20:13.260472 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:20:13.260456    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1\": container with ID starting with f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1 not found: ID does not exist" containerID="f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1"
Oct 30 23:20:13.260534 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.260476    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1"} err="failed to get container status \"f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1\": rpc error: code = NotFound desc = could not find container \"f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1\": container with ID starting with f6c0264503df6f3797fe98e6f4c8d5ae71bedea1b061479d49c4447fa18108c1 not found: ID does not exist"
Oct 30 23:20:13.260534 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.260490    5664 scope.go:122] "RemoveContainer" containerID="2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5"
Oct 30 23:20:13.260723 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:20:13.260708    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5\": container with ID starting with 2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5 not found: ID does not exist" containerID="2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5"
Oct 30 23:20:13.260762 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.260728    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5"} err="failed to get container status \"2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5\": rpc error: code = NotFound desc = could not find container \"2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5\": container with ID starting with 2a3c0d0b560348196b4c1e2c5a218a470c620f71ef1ba5a0e3333075629780a5 not found: ID does not exist"
Oct 30 23:20:13.273704 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.273681    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"]
Oct 30 23:20:13.292290 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.292270    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-7c457bbbbc-q6md2"]
Oct 30 23:20:13.867191 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:20:13.867160    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" path="/var/lib/kubelet/pods/efa1fd48-214b-4cbc-aa32-67700fee6163/volumes"
Oct 30 23:20:37.514755 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:20:37.514719    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:21:07.514530 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:21:07.514478    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:21:37.515427 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:21:37.515387    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:22:07.515410 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:22:07.515372    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:22:37.516161 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:22:37.516122    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:22:57.482357 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.482321    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"]
Oct 30 23:22:57.482874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.482637    5664 cpu_manager.go:414] "containerMap: removing container" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="storage-initializer"
Oct 30 23:22:57.482874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.482649    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="storage-initializer"
Oct 30 23:22:57.482874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.482662    5664 cpu_manager.go:414] "containerMap: removing container" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="llm-d-routing-sidecar"
Oct 30 23:22:57.482874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.482668    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="llm-d-routing-sidecar"
Oct 30 23:22:57.482874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.482688    5664 cpu_manager.go:414] "containerMap: removing container" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main"
Oct 30 23:22:57.482874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.482693    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main"
Oct 30 23:22:57.482874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.482743    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="main"
Oct 30 23:22:57.482874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.482754    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="efa1fd48-214b-4cbc-aa32-67700fee6163" containerName="llm-d-routing-sidecar"
Oct 30 23:22:57.485698 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.485681    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:22:57.490815 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.490793    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-with-lora-adapters-a290d4b0\"/\"seaweedfs-s3-creds\""
Oct 30 23:22:57.491057 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.491038    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-with-lora-adapters-a290d4b0\"/\"default-dockercfg-x7k84\""
Oct 30 23:22:57.491151 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.491139    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-with-lora-adapters-a290d4b0\"/\"kube-root-ca.crt\""
Oct 30 23:22:57.494606 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.494579    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"]
Oct 30 23:22:57.572437 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.572401    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmq7k\" (UniqueName: \"kubernetes.io/projected/f15e77c5-338d-4641-b0cc-e872746b028d-kube-api-access-xmq7k\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") " pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:22:57.591597 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.591566    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-with-lora-adapters-a290d4b0\"/\"openshift-service-ca.crt\""
Oct 30 23:22:57.673038 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.672994    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmq7k\" (UniqueName: \"kubernetes.io/projected/f15e77c5-338d-4641-b0cc-e872746b028d-kube-api-access-xmq7k\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") " pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:22:57.687043 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:22:57.687005    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmq7k\" (UniqueName: \"kubernetes.io/projected/f15e77c5-338d-4641-b0cc-e872746b028d-kube-api-access-xmq7k\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") " pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:00.702680 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:00.702640    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") " pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:00.710839 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:00.710811    5664 operation_generator.go:1464] "Controller attach succeeded for volume \"pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") device path: \"\"" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:00.804034 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:00.803996    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") " pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:00.804274 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:00.804142    5664 operation_generator.go:515] "MountVolume.WaitForAttach entering for volume \"pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") DevicePath \"\"" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:00.808457 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:00.808422    5664 operation_generator.go:525] "MountVolume.WaitForAttach succeeded for volume \"pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") DevicePath \"csi-e02e5ce1c753b2537ae455b3f7651b72f28e2d91bdacaafbe2cd86120269fe7f\"" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:01.024600 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:01.024565    5664 operation_generator.go:557] "MountVolume.MountDevice succeeded for volume \"pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") device mount path \"/var/lib/kubelet/plugins/kubernetes.io/csi/ebs.csi.aws.com/8f49b9622264ac8e6b1761dc84d8d30508642bcf7a845f93c1f2f860e39897e4/globalmount\"" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:01.039144 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:01.039112    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") pod \"e2e-pvc-lora-download-dcddf\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") " pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:01.097687 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:01.097647    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:01.372934 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:01.372707    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"]
Oct 30 23:23:01.375847 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:23:01.375813    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf15e77c5_338d_4641_b0cc_e872746b028d.slice/crio-b3fdc06bf58ccab3677450072a9147639ee7b014748d75c0fd6f46beaaba1bfa WatchSource:0}: Error finding container b3fdc06bf58ccab3677450072a9147639ee7b014748d75c0fd6f46beaaba1bfa: Status 404 returned error can't find the container with id b3fdc06bf58ccab3677450072a9147639ee7b014748d75c0fd6f46beaaba1bfa
Oct 30 23:23:01.907393 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:01.907352    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf" event={"ID":"f15e77c5-338d-4641-b0cc-e872746b028d","Type":"ContainerStarted","Data":"1839b6b194cb0da863a46f1dc9f2b3f1990fff3a035ae8e5bd604ba8e0c0ec3e"}
Oct 30 23:23:01.907835 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:01.907400    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf" event={"ID":"f15e77c5-338d-4641-b0cc-e872746b028d","Type":"ContainerStarted","Data":"b3fdc06bf58ccab3677450072a9147639ee7b014748d75c0fd6f46beaaba1bfa"}
Oct 30 23:23:01.929026 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:01.928953    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf" podStartSLOduration=7.928930135 podStartE2EDuration="7.928930135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 23:22:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:23:01.926756674 +0000 UTC m=+3812.178220167" watchObservedRunningTime="2026-10-30 23:23:01.928930135 +0000 UTC m=+3812.180393604"
Oct 30 23:23:03.918560 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:03.918469    5664 generic.go:356] "Generic (PLEG): container finished" podID="f15e77c5-338d-4641-b0cc-e872746b028d" containerID="1839b6b194cb0da863a46f1dc9f2b3f1990fff3a035ae8e5bd604ba8e0c0ec3e" exitCode=0
Oct 30 23:23:03.918560 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:03.918532    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf" event={"ID":"f15e77c5-338d-4641-b0cc-e872746b028d","Type":"ContainerDied","Data":"1839b6b194cb0da863a46f1dc9f2b3f1990fff3a035ae8e5bd604ba8e0c0ec3e"}
Oct 30 23:23:05.060819 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.060790    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:05.142574 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.142536    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") pod \"f15e77c5-338d-4641-b0cc-e872746b028d\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") "
Oct 30 23:23:05.142776 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.142623    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f15e77c5-338d-4641-b0cc-e872746b028d-kube-api-access-xmq7k\" (UniqueName: \"kubernetes.io/projected/f15e77c5-338d-4641-b0cc-e872746b028d-kube-api-access-xmq7k\") pod \"f15e77c5-338d-4641-b0cc-e872746b028d\" (UID: \"f15e77c5-338d-4641-b0cc-e872746b028d\") "
Oct 30 23:23:05.145023 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.144987    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f15e77c5-338d-4641-b0cc-e872746b028d-kube-api-access-xmq7k" pod "f15e77c5-338d-4641-b0cc-e872746b028d" (UID: "f15e77c5-338d-4641-b0cc-e872746b028d"). InnerVolumeSpecName "kube-api-access-xmq7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:23:05.146618 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.146588    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66" pod "f15e77c5-338d-4641-b0cc-e872746b028d" (UID: "f15e77c5-338d-4641-b0cc-e872746b028d"). InnerVolumeSpecName "pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad". PluginName "kubernetes.io/csi", VolumeGIDValue ""
Oct 30 23:23:05.243849 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.243748    5664 reconciler_common.go:292] "operationExecutor.UnmountDevice started for volume \"pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" "
Oct 30 23:23:05.243849 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.243790    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmq7k\" (UniqueName: \"kubernetes.io/projected/f15e77c5-338d-4641-b0cc-e872746b028d-kube-api-access-xmq7k\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:23:05.928297 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.928259    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf" event={"ID":"f15e77c5-338d-4641-b0cc-e872746b028d","Type":"ContainerDied","Data":"b3fdc06bf58ccab3677450072a9147639ee7b014748d75c0fd6f46beaaba1bfa"}
Oct 30 23:23:05.928297 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.928277    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"
Oct 30 23:23:05.928297 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:05.928302    5664 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3fdc06bf58ccab3677450072a9147639ee7b014748d75c0fd6f46beaaba1bfa"
Oct 30 23:23:06.219024 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:06.219002    5664 operation_generator.go:893] UnmountDevice succeeded for volume "pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad" (UniqueName: "kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66") on node "ip-10-0-132-192.us-east-2.compute.internal"
Oct 30 23:23:06.253395 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:06.253349    5664 reconciler_common.go:299] "Volume detached for volume \"pvc-f1c249b5-b7c9-4e8a-ade6-5bed0805d5ad\" (UniqueName: \"kubernetes.io/csi/ebs.csi.aws.com^vol-0bc959ddd15c69d66\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"csi-e02e5ce1c753b2537ae455b3f7651b72f28e2d91bdacaafbe2cd86120269fe7f\""
Oct 30 23:23:07.515730 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:23:07.515688    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:23:08.136463 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.136422    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"]
Oct 30 23:23:08.136805 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.136790    5664 cpu_manager.go:414] "containerMap: removing container" podUID="f15e77c5-338d-4641-b0cc-e872746b028d" containerName="storage-initializer"
Oct 30 23:23:08.136874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.136807    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f15e77c5-338d-4641-b0cc-e872746b028d" containerName="storage-initializer"
Oct 30 23:23:08.136874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.136860    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="f15e77c5-338d-4641-b0cc-e872746b028d" containerName="storage-initializer"
Oct 30 23:23:08.140141 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.140119    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.150746 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.150721    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:23:08.150746 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.150728    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Oct 30 23:23:08.150746 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.150748    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Oct 30 23:23:08.153392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.153366    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"seaweedfs-s3-creds\""
Oct 30 23:23:08.153554 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.153440    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-dockercfg-cc9jr\""
Oct 30 23:23:08.153628 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.153611    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-mnq7j\""
Oct 30 23:23:08.154293 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.153855    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Oct 30 23:23:08.155561 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.155537    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"]
Oct 30 23:23:08.273735 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.273688    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-home\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.273943 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.273748    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42wh7\" (UniqueName: \"kubernetes.io/projected/389b4ffa-4497-466c-89ed-2a3488954fa3-kube-api-access-42wh7\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.273943 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.273830    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/389b4ffa-4497-466c-89ed-2a3488954fa3-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.273943 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.273873    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.273943 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.273937    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-tmp-dir\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.274173 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.274002    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-model-cache\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.274173 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.274097    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/389b4ffa-4497-466c-89ed-2a3488954fa3-tls-certs\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.274173 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.274139    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-dshm\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375126 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375084    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/389b4ffa-4497-466c-89ed-2a3488954fa3-tls-certs\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375126 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375127    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-dshm\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375157    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-home\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375187    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42wh7\" (UniqueName: \"kubernetes.io/projected/389b4ffa-4497-466c-89ed-2a3488954fa3-kube-api-access-42wh7\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375216    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/389b4ffa-4497-466c-89ed-2a3488954fa3-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375234    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375262    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-tmp-dir\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375290    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-model-cache\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375735 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375608    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-home\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375735 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375625    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375827 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375735    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-model-cache\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.375865 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375852    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-tmp-dir\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.376008 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.375990    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/389b4ffa-4497-466c-89ed-2a3488954fa3-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.377706 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.377681    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-dshm\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.378078 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.378057    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/389b4ffa-4497-466c-89ed-2a3488954fa3-tls-certs\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.389576 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.389498    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42wh7\" (UniqueName: \"kubernetes.io/projected/389b4ffa-4497-466c-89ed-2a3488954fa3-kube-api-access-42wh7\") pod \"router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.458187 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.458148    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:08.756289 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.756184    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"]
Oct 30 23:23:08.758878 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:23:08.758852    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389b4ffa_4497_466c_89ed_2a3488954fa3.slice/crio-edc7f6bd045419803e0b2118c7469acdb203df6cfddb218d4d2b66a443dad90a WatchSource:0}: Error finding container edc7f6bd045419803e0b2118c7469acdb203df6cfddb218d4d2b66a443dad90a: Status 404 returned error can't find the container with id edc7f6bd045419803e0b2118c7469acdb203df6cfddb218d4d2b66a443dad90a
Oct 30 23:23:08.943221 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.943179    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" event={"ID":"389b4ffa-4497-466c-89ed-2a3488954fa3","Type":"ContainerStarted","Data":"b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a"}
Oct 30 23:23:08.943612 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.943251    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" event={"ID":"389b4ffa-4497-466c-89ed-2a3488954fa3","Type":"ContainerStarted","Data":"edc7f6bd045419803e0b2118c7469acdb203df6cfddb218d4d2b66a443dad90a"}
Oct 30 23:23:08.943612 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:08.943366    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:09.951600 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:09.951559    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" event={"ID":"389b4ffa-4497-466c-89ed-2a3488954fa3","Type":"ContainerStarted","Data":"1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b"}
Oct 30 23:23:11.961880 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:11.961838    5664 generic.go:356] "Generic (PLEG): container finished" podID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerID="1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b" exitCode=0
Oct 30 23:23:11.962350 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:11.961927    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" event={"ID":"389b4ffa-4497-466c-89ed-2a3488954fa3","Type":"ContainerDied","Data":"1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b"}
Oct 30 23:23:12.969324 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:12.969282    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" event={"ID":"389b4ffa-4497-466c-89ed-2a3488954fa3","Type":"ContainerStarted","Data":"65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab"}
Oct 30 23:23:13.001225 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:13.001163    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podStartSLOduration=2.7077260819999998 podStartE2EDuration="5.001146272s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.29342019s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:23:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:23:12.99967204 +0000 UTC m=+3823.251135509" watchObservedRunningTime="2026-10-30 23:23:13.001146272 +0000 UTC m=+3823.252609765"
Oct 30 23:23:18.458951 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:18.458871    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:18.459512 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:18.459033    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:18.460492 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:18.460456    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:23:18.472283 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:18.472255    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:23:28.462934 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:28.461001    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:23:37.515318 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:23:37.515273    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:23:38.459194 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:38.459148    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:23:46.263438 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:46.263367    5664 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal" podUID="320724d28f4bb4eb7790f5cce736ad83" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Oct 30 23:23:48.458755 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:48.458691    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:23:58.459113 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:23:58.459060    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:24:07.517381 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:24:07.517337    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:24:08.459069 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:24:08.459014    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:24:18.458810 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:24:18.458694    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:24:28.459129 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:24:28.459079    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:24:37.517680 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:24:37.517637    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:24:38.459031 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:24:38.458984    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:24:48.458593 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:24:48.458536    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:24:58.458933 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:24:58.458854    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:25:07.516463 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:25:07.516416    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:25:08.458891 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:08.458816    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8001/health\": dial tcp 10.134.0.43:8001: connect: connection refused"
Oct 30 23:25:18.468309 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:18.468278    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:25:18.480107 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:18.480081    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:25:37.515677 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:25:37.515638    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:25:40.452911 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:40.452811    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"]
Oct 30 23:25:55.476863 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:55.476794    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main" containerID="cri-o://65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab" gracePeriod=45
Oct 30 23:25:55.988756 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:55.988704    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="llm-d-routing-sidecar" containerID="cri-o://b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a" gracePeriod=45
Oct 30 23:25:56.226618 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.226592    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:25:56.244781 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.244754    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/389b4ffa-4497-466c-89ed-2a3488954fa3-tls-certs\" (UniqueName: \"kubernetes.io/secret/389b4ffa-4497-466c-89ed-2a3488954fa3-tls-certs\") pod \"389b4ffa-4497-466c-89ed-2a3488954fa3\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") "
Oct 30 23:25:56.244933 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.244792    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-home\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-home\") pod \"389b4ffa-4497-466c-89ed-2a3488954fa3\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") "
Oct 30 23:25:56.244933 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.244822    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/389b4ffa-4497-466c-89ed-2a3488954fa3-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/389b4ffa-4497-466c-89ed-2a3488954fa3-cabundle-cert\") pod \"389b4ffa-4497-466c-89ed-2a3488954fa3\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") "
Oct 30 23:25:56.244933 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.244841    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-tmp-dir\") pod \"389b4ffa-4497-466c-89ed-2a3488954fa3\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") "
Oct 30 23:25:56.244933 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.244878    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/389b4ffa-4497-466c-89ed-2a3488954fa3-kube-api-access-42wh7\" (UniqueName: \"kubernetes.io/projected/389b4ffa-4497-466c-89ed-2a3488954fa3-kube-api-access-42wh7\") pod \"389b4ffa-4497-466c-89ed-2a3488954fa3\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") "
Oct 30 23:25:56.245164 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.244947    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-model-cache\") pod \"389b4ffa-4497-466c-89ed-2a3488954fa3\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") "
Oct 30 23:25:56.245164 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.244972    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-dshm\") pod \"389b4ffa-4497-466c-89ed-2a3488954fa3\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") "
Oct 30 23:25:56.245164 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.245010    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-kserve-provision-location\") pod \"389b4ffa-4497-466c-89ed-2a3488954fa3\" (UID: \"389b4ffa-4497-466c-89ed-2a3488954fa3\") "
Oct 30 23:25:56.245327 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.245255    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-model-cache" pod "389b4ffa-4497-466c-89ed-2a3488954fa3" (UID: "389b4ffa-4497-466c-89ed-2a3488954fa3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:25:56.245383 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.245350    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-home" pod "389b4ffa-4497-466c-89ed-2a3488954fa3" (UID: "389b4ffa-4497-466c-89ed-2a3488954fa3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:25:56.245685 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.245498    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/389b4ffa-4497-466c-89ed-2a3488954fa3-cabundle-cert" pod "389b4ffa-4497-466c-89ed-2a3488954fa3" (UID: "389b4ffa-4497-466c-89ed-2a3488954fa3"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:25:56.247356 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.247315    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/389b4ffa-4497-466c-89ed-2a3488954fa3-tls-certs" pod "389b4ffa-4497-466c-89ed-2a3488954fa3" (UID: "389b4ffa-4497-466c-89ed-2a3488954fa3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:25:56.247471 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.247402    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/389b4ffa-4497-466c-89ed-2a3488954fa3-kube-api-access-42wh7" pod "389b4ffa-4497-466c-89ed-2a3488954fa3" (UID: "389b4ffa-4497-466c-89ed-2a3488954fa3"). InnerVolumeSpecName "kube-api-access-42wh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:25:56.247471 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.247413    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-dshm" pod "389b4ffa-4497-466c-89ed-2a3488954fa3" (UID: "389b4ffa-4497-466c-89ed-2a3488954fa3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:25:56.265221 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.265187    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-tmp-dir" pod "389b4ffa-4497-466c-89ed-2a3488954fa3" (UID: "389b4ffa-4497-466c-89ed-2a3488954fa3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:25:56.277199 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.277172    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-kserve-provision-location" pod "389b4ffa-4497-466c-89ed-2a3488954fa3" (UID: "389b4ffa-4497-466c-89ed-2a3488954fa3"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:25:56.345650 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.345621    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/389b4ffa-4497-466c-89ed-2a3488954fa3-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:25:56.345650 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.345647    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:25:56.345791 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.345659    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/389b4ffa-4497-466c-89ed-2a3488954fa3-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:25:56.345791 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.345670    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:25:56.345791 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.345680    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-42wh7\" (UniqueName: \"kubernetes.io/projected/389b4ffa-4497-466c-89ed-2a3488954fa3-kube-api-access-42wh7\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:25:56.345791 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.345690    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:25:56.345791 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.345698    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:25:56.345791 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.345707    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/389b4ffa-4497-466c-89ed-2a3488954fa3-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:25:56.717074 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.717041    5664 generic.go:356] "Generic (PLEG): container finished" podID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerID="65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab" exitCode=0
Oct 30 23:25:56.717074 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.717070    5664 generic.go:356] "Generic (PLEG): container finished" podID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerID="b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a" exitCode=0
Oct 30 23:25:56.717526 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.717102    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"
Oct 30 23:25:56.717526 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.717128    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" event={"ID":"389b4ffa-4497-466c-89ed-2a3488954fa3","Type":"ContainerDied","Data":"65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab"}
Oct 30 23:25:56.717526 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.717190    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" event={"ID":"389b4ffa-4497-466c-89ed-2a3488954fa3","Type":"ContainerDied","Data":"b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a"}
Oct 30 23:25:56.717526 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.717217    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4" event={"ID":"389b4ffa-4497-466c-89ed-2a3488954fa3","Type":"ContainerDied","Data":"edc7f6bd045419803e0b2118c7469acdb203df6cfddb218d4d2b66a443dad90a"}
Oct 30 23:25:56.717526 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.717249    5664 scope.go:122] "RemoveContainer" containerID="65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab"
Oct 30 23:25:56.726386 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.726359    5664 scope.go:122] "RemoveContainer" containerID="1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b"
Oct 30 23:25:56.734334 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.734317    5664 scope.go:122] "RemoveContainer" containerID="b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a"
Oct 30 23:25:56.741368 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.741348    5664 scope.go:122] "RemoveContainer" containerID="65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab"
Oct 30 23:25:56.741628 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:25:56.741607    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab\": container with ID starting with 65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab not found: ID does not exist" containerID="65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab"
Oct 30 23:25:56.741726 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.741640    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab"} err="failed to get container status \"65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab\": rpc error: code = NotFound desc = could not find container \"65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab\": container with ID starting with 65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab not found: ID does not exist"
Oct 30 23:25:56.741726 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.741662    5664 scope.go:122] "RemoveContainer" containerID="1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b"
Oct 30 23:25:56.742078 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:25:56.741991    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b\": container with ID starting with 1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b not found: ID does not exist" containerID="1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b"
Oct 30 23:25:56.742078 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.742032    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b"} err="failed to get container status \"1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b\": rpc error: code = NotFound desc = could not find container \"1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b\": container with ID starting with 1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b not found: ID does not exist"
Oct 30 23:25:56.742078 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.742046    5664 scope.go:122] "RemoveContainer" containerID="b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a"
Oct 30 23:25:56.742372 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:25:56.742349    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a\": container with ID starting with b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a not found: ID does not exist" containerID="b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a"
Oct 30 23:25:56.742472 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.742376    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a"} err="failed to get container status \"b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a\": rpc error: code = NotFound desc = could not find container \"b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a\": container with ID starting with b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a not found: ID does not exist"
Oct 30 23:25:56.742472 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.742396    5664 scope.go:122] "RemoveContainer" containerID="65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab"
Oct 30 23:25:56.742662 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.742640    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab"} err="failed to get container status \"65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab\": rpc error: code = NotFound desc = could not find container \"65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab\": container with ID starting with 65b6de4ea89e601d428f7c7d551b258f011bb0aa9b03952f982e6f8feba2f5ab not found: ID does not exist"
Oct 30 23:25:56.742732 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.742664    5664 scope.go:122] "RemoveContainer" containerID="1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b"
Oct 30 23:25:56.742948 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.742925    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b"} err="failed to get container status \"1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b\": rpc error: code = NotFound desc = could not find container \"1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b\": container with ID starting with 1b5735e3ac72ca536812dbb75523697f11410f38d45a47f2a3b06c9eef4fc04b not found: ID does not exist"
Oct 30 23:25:56.743036 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.742951    5664 scope.go:122] "RemoveContainer" containerID="b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a"
Oct 30 23:25:56.743256 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.743230    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a"} err="failed to get container status \"b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a\": rpc error: code = NotFound desc = could not find container \"b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a\": container with ID starting with b496d878d9751d5df03a37e45de1a22ab8f450008c09a0423217891a4e01219a not found: ID does not exist"
Oct 30 23:25:56.745021 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.744998    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"]
Oct 30 23:25:56.746471 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:56.746449    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-7cb5b49cb5-s49p4"]
Oct 30 23:25:57.868150 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:25:57.868116    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" path="/var/lib/kubelet/pods/389b4ffa-4497-466c-89ed-2a3488954fa3/volumes"
Oct 30 23:26:03.687182 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.687149    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"]
Oct 30 23:26:03.687598 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.687447    5664 cpu_manager.go:414] "containerMap: removing container" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main"
Oct 30 23:26:03.687598 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.687458    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main"
Oct 30 23:26:03.687598 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.687477    5664 cpu_manager.go:414] "containerMap: removing container" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="storage-initializer"
Oct 30 23:26:03.687598 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.687482    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="storage-initializer"
Oct 30 23:26:03.687598 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.687514    5664 cpu_manager.go:414] "containerMap: removing container" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="llm-d-routing-sidecar"
Oct 30 23:26:03.687598 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.687520    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="llm-d-routing-sidecar"
Oct 30 23:26:03.687598 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.687573    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="llm-d-routing-sidecar"
Oct 30 23:26:03.687598 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.687582    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="389b4ffa-4497-466c-89ed-2a3488954fa3" containerName="main"
Oct 30 23:26:03.692344 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.692326    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.693866 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.693843    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-tmp-dir\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.693993 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.693880    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptg6q\" (UniqueName: \"kubernetes.io/projected/befe052f-c396-4447-8fff-a63d430c6584-kube-api-access-ptg6q\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.694042 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.693986    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/befe052f-c396-4447-8fff-a63d430c6584-tls-certs\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.694042 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.694028    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-model-cache\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.694149 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.694060    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-dshm\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.694149 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.694080    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-home\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.697999 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.697978    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"default-dockercfg-9jc8l\""
Oct 30 23:26:03.698102 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.698007    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"openshift-service-ca.crt\""
Oct 30 23:26:03.698102 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.698024    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"kube-root-ca.crt\""
Oct 30 23:26:03.698185 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.698102    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-model-based-routing-only-f772d59b\"/\"model-routing-only-7aad4d71-kserve-self-signed-certs\""
Oct 30 23:26:03.702260 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.702239    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"]
Oct 30 23:26:03.794845 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.794807    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-tmp-dir\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.794845 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.794846    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptg6q\" (UniqueName: \"kubernetes.io/projected/befe052f-c396-4447-8fff-a63d430c6584-kube-api-access-ptg6q\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.795137 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.794872    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/befe052f-c396-4447-8fff-a63d430c6584-tls-certs\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.795137 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.794924    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-model-cache\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.795137 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.794953    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-dshm\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.795137 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.794979    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-home\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.795311 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.795270    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-tmp-dir\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.795348 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.795306    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-model-cache\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.795348 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.795321    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-home\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.797457 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.797431    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-dshm\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.797564 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.797504    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/befe052f-c396-4447-8fff-a63d430c6584-tls-certs\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:03.808388 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:03.808362    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptg6q\" (UniqueName: \"kubernetes.io/projected/befe052f-c396-4447-8fff-a63d430c6584-kube-api-access-ptg6q\") pod \"model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") " pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:04.005395 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:04.005309    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:04.166107 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:04.166073    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"]
Oct 30 23:26:04.169359 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:04.169331    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-with-lora-adapters-a290d4b0/e2e-pvc-lora-download-dcddf"]
Oct 30 23:26:04.279227 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:04.279090    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"]
Oct 30 23:26:04.281848 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:26:04.281818    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbefe052f_c396_4447_8fff_a63d430c6584.slice/crio-6e08b7f3804016b521ff5d8a2ec69c33463ab8d94f2a9d559ea6c1602b7dcd86 WatchSource:0}: Error finding container 6e08b7f3804016b521ff5d8a2ec69c33463ab8d94f2a9d559ea6c1602b7dcd86: Status 404 returned error can't find the container with id 6e08b7f3804016b521ff5d8a2ec69c33463ab8d94f2a9d559ea6c1602b7dcd86
Oct 30 23:26:04.753073 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:04.753020    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9" event={"ID":"befe052f-c396-4447-8fff-a63d430c6584","Type":"ContainerStarted","Data":"85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2"}
Oct 30 23:26:04.753461 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:04.753093    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9" event={"ID":"befe052f-c396-4447-8fff-a63d430c6584","Type":"ContainerStarted","Data":"6e08b7f3804016b521ff5d8a2ec69c33463ab8d94f2a9d559ea6c1602b7dcd86"}
Oct 30 23:26:04.773793 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:04.773739    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9" podStartSLOduration=1.7737222209999999 podStartE2EDuration="1.773722221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:26:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:26:04.773277431 +0000 UTC m=+3995.024740923" watchObservedRunningTime="2026-10-30 23:26:04.773722221 +0000 UTC m=+3995.025185681"
Oct 30 23:26:05.005854 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:05.005765    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:05.867678 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:05.867641    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f15e77c5-338d-4641-b0cc-e872746b028d" path="/var/lib/kubelet/pods/f15e77c5-338d-4641-b0cc-e872746b028d/volumes"
Oct 30 23:26:07.514867 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:26:07.514829    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:26:14.005678 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:14.005629    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:14.018135 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:14.018110    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:14.805584 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:26:14.805555    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:26:37.514983 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:26:37.514944    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:27:07.514961 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:27:07.514911    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:27:21.130299 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.130259    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"]
Oct 30 23:27:21.140172 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:27:21.140130    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2" cmd=["/bin/sleep","15"]
Oct 30 23:27:21.140317 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.140193    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 23:27:21.140317 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:27:21.140206    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9" podUID="befe052f-c396-4447-8fff-a63d430c6584" containerName="main" containerID="cri-o://85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2"
Oct 30 23:27:21.140317 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.140251    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9" podUID="befe052f-c396-4447-8fff-a63d430c6584" containerName="main" containerID="cri-o://85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2" gracePeriod=60
Oct 30 23:27:21.490994 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.490968    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:27:21.548409 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.548377    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/befe052f-c396-4447-8fff-a63d430c6584-tls-certs\" (UniqueName: \"kubernetes.io/secret/befe052f-c396-4447-8fff-a63d430c6584-tls-certs\") pod \"befe052f-c396-4447-8fff-a63d430c6584\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") "
Oct 30 23:27:21.548599 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.548444    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-dshm\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-dshm\") pod \"befe052f-c396-4447-8fff-a63d430c6584\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") "
Oct 30 23:27:21.548599 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.548507    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-home\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-home\") pod \"befe052f-c396-4447-8fff-a63d430c6584\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") "
Oct 30 23:27:21.548599 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.548534    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-model-cache\") pod \"befe052f-c396-4447-8fff-a63d430c6584\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") "
Oct 30 23:27:21.548599 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.548586    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-tmp-dir\") pod \"befe052f-c396-4447-8fff-a63d430c6584\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") "
Oct 30 23:27:21.548814 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.548645    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/befe052f-c396-4447-8fff-a63d430c6584-kube-api-access-ptg6q\" (UniqueName: \"kubernetes.io/projected/befe052f-c396-4447-8fff-a63d430c6584-kube-api-access-ptg6q\") pod \"befe052f-c396-4447-8fff-a63d430c6584\" (UID: \"befe052f-c396-4447-8fff-a63d430c6584\") "
Oct 30 23:27:21.548866 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.548842    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-model-cache" pod "befe052f-c396-4447-8fff-a63d430c6584" (UID: "befe052f-c396-4447-8fff-a63d430c6584"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:27:21.548866 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.548858    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-home" pod "befe052f-c396-4447-8fff-a63d430c6584" (UID: "befe052f-c396-4447-8fff-a63d430c6584"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:27:21.549045 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.549015    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-tmp-dir" pod "befe052f-c396-4447-8fff-a63d430c6584" (UID: "befe052f-c396-4447-8fff-a63d430c6584"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:27:21.551027 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.550869    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/befe052f-c396-4447-8fff-a63d430c6584-tls-certs" pod "befe052f-c396-4447-8fff-a63d430c6584" (UID: "befe052f-c396-4447-8fff-a63d430c6584"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:27:21.551156 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.551094    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/befe052f-c396-4447-8fff-a63d430c6584-kube-api-access-ptg6q" pod "befe052f-c396-4447-8fff-a63d430c6584" (UID: "befe052f-c396-4447-8fff-a63d430c6584"). InnerVolumeSpecName "kube-api-access-ptg6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:27:21.551203 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.551173    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-dshm" pod "befe052f-c396-4447-8fff-a63d430c6584" (UID: "befe052f-c396-4447-8fff-a63d430c6584"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:27:21.649585 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.649551    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:27:21.649585 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.649582    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:27:21.649802 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.649594    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:27:21.649802 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.649608    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/befe052f-c396-4447-8fff-a63d430c6584-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:27:21.649802 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.649620    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptg6q\" (UniqueName: \"kubernetes.io/projected/befe052f-c396-4447-8fff-a63d430c6584-kube-api-access-ptg6q\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:27:21.649802 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:21.649633    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/befe052f-c396-4447-8fff-a63d430c6584-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:27:22.062510 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:22.062475    5664 generic.go:356] "Generic (PLEG): container finished" podID="befe052f-c396-4447-8fff-a63d430c6584" containerID="85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2" exitCode=0
Oct 30 23:27:22.062693 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:22.062518    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"
Oct 30 23:27:22.062693 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:22.062556    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9" event={"ID":"befe052f-c396-4447-8fff-a63d430c6584","Type":"ContainerDied","Data":"85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2"}
Oct 30 23:27:22.062693 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:22.062613    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9" event={"ID":"befe052f-c396-4447-8fff-a63d430c6584","Type":"ContainerDied","Data":"6e08b7f3804016b521ff5d8a2ec69c33463ab8d94f2a9d559ea6c1602b7dcd86"}
Oct 30 23:27:22.062693 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:22.062635    5664 scope.go:122] "RemoveContainer" containerID="85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2"
Oct 30 23:27:22.071311 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:22.071292    5664 scope.go:122] "RemoveContainer" containerID="85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2"
Oct 30 23:27:22.071633 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:27:22.071611    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2\": container with ID starting with 85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2 not found: ID does not exist" containerID="85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2"
Oct 30 23:27:22.071719 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:22.071643    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2"} err="failed to get container status \"85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2\": rpc error: code = NotFound desc = could not find container \"85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2\": container with ID starting with 85ab3081d49700fbe3167f054b28cbf673149a337f92ae538dcb7576be4f5fa2 not found: ID does not exist"
Oct 30 23:27:22.082666 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:22.082638    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"]
Oct 30 23:27:22.087213 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:22.087191    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-model-based-routing-only-f772d59b/model-routing-only-7aad4d71-kserve-6557db66dd-jsnr9"]
Oct 30 23:27:23.867054 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:27:23.867024    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="befe052f-c396-4447-8fff-a63d430c6584" path="/var/lib/kubelet/pods/befe052f-c396-4447-8fff-a63d430c6584/volumes"
Oct 30 23:27:37.515417 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:27:37.515374    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:28:07.514599 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:28:07.514559    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:28:37.515094 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:28:37.515055    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:28:48.895818 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.893708    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"]
Oct 30 23:28:48.895818 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.894245    5664 cpu_manager.go:414] "containerMap: removing container" podUID="befe052f-c396-4447-8fff-a63d430c6584" containerName="main"
Oct 30 23:28:48.895818 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.894266    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="befe052f-c396-4447-8fff-a63d430c6584" containerName="main"
Oct 30 23:28:48.895818 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.894361    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="befe052f-c396-4447-8fff-a63d430c6584" containerName="main"
Oct 30 23:28:48.899225 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.898144    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:48.912935 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.912183    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"default-dockercfg-fjgbt\""
Oct 30 23:28:48.912935 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.912303    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"tls-verification-test-kserve-dockercfg-h85jx\""
Oct 30 23:28:48.912935 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.912418    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"tls-verification-test-kserve-self-signed-certs\""
Oct 30 23:28:48.916393 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.914013    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"]
Oct 30 23:28:48.916777 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.916752    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"openshift-service-ca.crt\""
Oct 30 23:28:48.917457 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.917299    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:28:48.917620 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.917393    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"kube-root-ca.crt\""
Oct 30 23:28:48.927568 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.918697    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-model-cache\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:48.927568 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.918757    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tmp-dir\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:48.927568 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.918783    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-home\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:48.927568 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.918836    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tls-certs\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:48.927568 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.918890    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kserve-provision-location\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:48.927568 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.918942    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-dshm\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:48.927568 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.918986    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/053dde5c-1ffd-47c4-8a1c-85c7187dc621-cabundle-cert\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:48.927568 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:48.919013    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wck6v\" (UniqueName: \"kubernetes.io/projected/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kube-api-access-wck6v\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.008226 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.008173    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-tls-resources-e4cad0c3\"/\"seaweedfs-s3-creds\""
Oct 30 23:28:49.030712 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.030668    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-model-cache\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.031097 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031058    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tmp-dir\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.031260 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031244    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-home\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031358    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-model-cache\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031410    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tls-certs\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031493    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kserve-provision-location\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031546    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-dshm\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031601    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/053dde5c-1ffd-47c4-8a1c-85c7187dc621-cabundle-cert\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031621    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tmp-dir\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031630    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wck6v\" (UniqueName: \"kubernetes.io/projected/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kube-api-access-wck6v\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.031871    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kserve-provision-location\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.032142    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-home\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.032942 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.032743    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/053dde5c-1ffd-47c4-8a1c-85c7187dc621-cabundle-cert\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.060824 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.060746    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tls-certs\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.063349 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.063315    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-dshm\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.064801 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.064704    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wck6v\" (UniqueName: \"kubernetes.io/projected/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kube-api-access-wck6v\") pod \"tls-verification-test-kserve-656f77bc69-6b9c8\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") " pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.271062 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.271025    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:49.756869 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:49.756821    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"]
Oct 30 23:28:49.761433 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:28:49.761400    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod053dde5c_1ffd_47c4_8a1c_85c7187dc621.slice/crio-7bc617fc9aa5bdf4f5d1072e5fadb299aa6a2e17be4ddd6baf532b6574271f22 WatchSource:0}: Error finding container 7bc617fc9aa5bdf4f5d1072e5fadb299aa6a2e17be4ddd6baf532b6574271f22: Status 404 returned error can't find the container with id 7bc617fc9aa5bdf4f5d1072e5fadb299aa6a2e17be4ddd6baf532b6574271f22
Oct 30 23:28:50.477718 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:50.477684    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" event={"ID":"053dde5c-1ffd-47c4-8a1c-85c7187dc621","Type":"ContainerStarted","Data":"0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6"}
Oct 30 23:28:50.478167 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:50.477742    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" event={"ID":"053dde5c-1ffd-47c4-8a1c-85c7187dc621","Type":"ContainerStarted","Data":"7bc617fc9aa5bdf4f5d1072e5fadb299aa6a2e17be4ddd6baf532b6574271f22"}
Oct 30 23:28:50.478167 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:50.477785    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:51.484551 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:51.484517    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" event={"ID":"053dde5c-1ffd-47c4-8a1c-85c7187dc621","Type":"ContainerStarted","Data":"2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259"}
Oct 30 23:28:53.495863 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:53.495832    5664 generic.go:356] "Generic (PLEG): container finished" podID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerID="2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259" exitCode=0
Oct 30 23:28:53.496284 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:53.495926    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" event={"ID":"053dde5c-1ffd-47c4-8a1c-85c7187dc621","Type":"ContainerDied","Data":"2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259"}
Oct 30 23:28:54.502220 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:54.502184    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" event={"ID":"053dde5c-1ffd-47c4-8a1c-85c7187dc621","Type":"ContainerStarted","Data":"58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a"}
Oct 30 23:28:54.536480 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:54.536413    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podStartSLOduration=4.143959709 podStartE2EDuration="6.536397509s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.3924378s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:28:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:28:54.534948793 +0000 UTC m=+4164.786412261" watchObservedRunningTime="2026-10-30 23:28:54.536397509 +0000 UTC m=+4164.787860976"
Oct 30 23:28:59.272232 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:59.272189    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:59.272678 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:59.272280    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:28:59.273482 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:59.273451    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:28:59.285249 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:28:59.285227    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:29:07.514378 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:29:07.514337    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:29:09.271619 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:29:09.271582    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:29:19.272633 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:29:19.272550    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:29:29.271651 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:29:29.271603    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:29:30.388410 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:29:30.388376    5664 scope.go:122] "RemoveContainer" containerID="1839b6b194cb0da863a46f1dc9f2b3f1990fff3a035ae8e5bd604ba8e0c0ec3e"
Oct 30 23:29:37.516519 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:29:37.516476    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:29:39.271623 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:29:39.271584    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:29:49.271916 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:29:49.271862    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:29:59.271994 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:29:59.271940    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:30:07.516053 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:30:07.516006    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:30:09.272652 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:30:09.272599    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:30:19.271918 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:30:19.271855    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:30:29.271806 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:30:29.271760    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:30:37.515916 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:30:37.515863    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:30:39.272384 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:30:39.272338    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8001/health\": dial tcp 10.134.0.45:8001: connect: connection refused"
Oct 30 23:30:49.287538 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:30:49.287504    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:30:49.299304 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:30:49.299278    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:31:07.515081 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:31:07.515040    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:31:37.515882 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:31:37.515841    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:32:07.515929 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:32:07.515877    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:32:37.516680 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:32:37.516641    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:33:07.517644 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:33:07.517599    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:33:37.515298 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:33:37.515257    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:33:47.231964 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:33:47.231922    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-tls-resources-e4cad0c3_tls-verification-test-kserve-656f77bc69-6b9c8_053dde5c-1ffd-47c4-8a1c-85c7187dc621/llm-d-routing-sidecar/0.log"
Oct 30 23:33:47.254193 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:33:47.254163    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-tls-resources-e4cad0c3_tls-verification-test-kserve-656f77bc69-6b9c8_053dde5c-1ffd-47c4-8a1c-85c7187dc621/storage-initializer/0.log"
Oct 30 23:33:47.278360 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:33:47.278333    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-llm-tls-resources-e4cad0c3_tls-verification-test-kserve-656f77bc69-6b9c8_053dde5c-1ffd-47c4-8a1c-85c7187dc621/main/0.log"
Oct 30 23:33:51.151679 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:33:51.151647    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"]
Oct 30 23:34:06.174614 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:06.174527    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main" containerID="cri-o://58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a" gracePeriod=45
Oct 30 23:34:06.791766 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:06.791703    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="llm-d-routing-sidecar" containerID="cri-o://0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6" gracePeriod=45
Oct 30 23:34:06.800708 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:06.800682    5664 generic.go:356] "Generic (PLEG): container finished" podID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerID="58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a" exitCode=0
Oct 30 23:34:06.800805 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:06.800758    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" event={"ID":"053dde5c-1ffd-47c4-8a1c-85c7187dc621","Type":"ContainerDied","Data":"58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a"}
Oct 30 23:34:07.023606 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.023579    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:34:07.111135 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111098    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-home\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-home\") pod \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") "
Oct 30 23:34:07.111316 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111150    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-dshm\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-dshm\") pod \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") "
Oct 30 23:34:07.111316 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111178    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kserve-provision-location\") pod \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") "
Oct 30 23:34:07.111316 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111199    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-model-cache\") pod \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") "
Oct 30 23:34:07.111316 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111219    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tls-certs\" (UniqueName: \"kubernetes.io/secret/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tls-certs\") pod \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") "
Oct 30 23:34:07.111316 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111245    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kube-api-access-wck6v\" (UniqueName: \"kubernetes.io/projected/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kube-api-access-wck6v\") pod \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") "
Oct 30 23:34:07.111316 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111307    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tmp-dir\") pod \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") "
Oct 30 23:34:07.111638 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111327    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/053dde5c-1ffd-47c4-8a1c-85c7187dc621-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/053dde5c-1ffd-47c4-8a1c-85c7187dc621-cabundle-cert\") pod \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\" (UID: \"053dde5c-1ffd-47c4-8a1c-85c7187dc621\") "
Oct 30 23:34:07.111638 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111498    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-model-cache" pod "053dde5c-1ffd-47c4-8a1c-85c7187dc621" (UID: "053dde5c-1ffd-47c4-8a1c-85c7187dc621"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:07.111968 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111928    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-home" pod "053dde5c-1ffd-47c4-8a1c-85c7187dc621" (UID: "053dde5c-1ffd-47c4-8a1c-85c7187dc621"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:07.111968 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.111940    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/053dde5c-1ffd-47c4-8a1c-85c7187dc621-cabundle-cert" pod "053dde5c-1ffd-47c4-8a1c-85c7187dc621" (UID: "053dde5c-1ffd-47c4-8a1c-85c7187dc621"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:34:07.113753 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.113728    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kube-api-access-wck6v" pod "053dde5c-1ffd-47c4-8a1c-85c7187dc621" (UID: "053dde5c-1ffd-47c4-8a1c-85c7187dc621"). InnerVolumeSpecName "kube-api-access-wck6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:07.113869 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.113818    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-dshm" pod "053dde5c-1ffd-47c4-8a1c-85c7187dc621" (UID: "053dde5c-1ffd-47c4-8a1c-85c7187dc621"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:07.113970 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.113863    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tls-certs" pod "053dde5c-1ffd-47c4-8a1c-85c7187dc621" (UID: "053dde5c-1ffd-47c4-8a1c-85c7187dc621"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:34:07.123434 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.123413    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tmp-dir" pod "053dde5c-1ffd-47c4-8a1c-85c7187dc621" (UID: "053dde5c-1ffd-47c4-8a1c-85c7187dc621"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:07.133413 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.133393    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kserve-provision-location" pod "053dde5c-1ffd-47c4-8a1c-85c7187dc621" (UID: "053dde5c-1ffd-47c4-8a1c-85c7187dc621"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:07.212727 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.212660    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:07.212727 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.212683    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/053dde5c-1ffd-47c4-8a1c-85c7187dc621-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:07.212727 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.212695    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:07.212727 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.212703    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:07.212727 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.212711    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:07.212727 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.212721    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/053dde5c-1ffd-47c4-8a1c-85c7187dc621-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:07.212727 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.212729    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/053dde5c-1ffd-47c4-8a1c-85c7187dc621-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:07.213270 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.212736    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wck6v\" (UniqueName: \"kubernetes.io/projected/053dde5c-1ffd-47c4-8a1c-85c7187dc621-kube-api-access-wck6v\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:07.514847 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:34:07.514752    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:34:07.807401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.807304    5664 generic.go:356] "Generic (PLEG): container finished" podID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerID="0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6" exitCode=0
Oct 30 23:34:07.807401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.807350    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" event={"ID":"053dde5c-1ffd-47c4-8a1c-85c7187dc621","Type":"ContainerDied","Data":"0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6"}
Oct 30 23:34:07.807401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.807371    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"
Oct 30 23:34:07.807401 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.807394    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8" event={"ID":"053dde5c-1ffd-47c4-8a1c-85c7187dc621","Type":"ContainerDied","Data":"7bc617fc9aa5bdf4f5d1072e5fadb299aa6a2e17be4ddd6baf532b6574271f22"}
Oct 30 23:34:07.807736 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.807415    5664 scope.go:122] "RemoveContainer" containerID="58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a"
Oct 30 23:34:07.816841 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.816821    5664 scope.go:122] "RemoveContainer" containerID="2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259"
Oct 30 23:34:07.824185 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.824163    5664 scope.go:122] "RemoveContainer" containerID="0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6"
Oct 30 23:34:07.831819 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.831795    5664 scope.go:122] "RemoveContainer" containerID="58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a"
Oct 30 23:34:07.832164 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:34:07.832142    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a\": container with ID starting with 58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a not found: ID does not exist" containerID="58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a"
Oct 30 23:34:07.832297 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.832270    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a"} err="failed to get container status \"58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a\": rpc error: code = NotFound desc = could not find container \"58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a\": container with ID starting with 58528918074e987e9cfb7a99cf37e777cbc4a8130ed6fe683d89b67ed00f7d3a not found: ID does not exist"
Oct 30 23:34:07.832420 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.832300    5664 scope.go:122] "RemoveContainer" containerID="2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259"
Oct 30 23:34:07.832480 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.832412    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"]
Oct 30 23:34:07.832726 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:34:07.832601    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259\": container with ID starting with 2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259 not found: ID does not exist" containerID="2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259"
Oct 30 23:34:07.832726 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.832633    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259"} err="failed to get container status \"2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259\": rpc error: code = NotFound desc = could not find container \"2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259\": container with ID starting with 2e91ed9f4504930bd8b725646fee1586394c1b20bc5e8f1f7a5cbb4e69bd1259 not found: ID does not exist"
Oct 30 23:34:07.832726 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.832653    5664 scope.go:122] "RemoveContainer" containerID="0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6"
Oct 30 23:34:07.833005 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:34:07.832983    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6\": container with ID starting with 0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6 not found: ID does not exist" containerID="0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6"
Oct 30 23:34:07.833066 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.833013    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6"} err="failed to get container status \"0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6\": rpc error: code = NotFound desc = could not find container \"0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6\": container with ID starting with 0b92c118c2e5c6a75763a8349f6b2efddba35fb0805d2533c5f7349cf41b40d6 not found: ID does not exist"
Oct 30 23:34:07.840324 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.840303    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-tls-resources-e4cad0c3/tls-verification-test-kserve-656f77bc69-6b9c8"]
Oct 30 23:34:07.867666 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:07.867640    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" path="/var/lib/kubelet/pods/053dde5c-1ffd-47c4-8a1c-85c7187dc621/volumes"
Oct 30 23:34:11.463577 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:11.463537    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"]
Oct 30 23:34:11.465185 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:11.463797    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy" containerID="cri-o://d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2" gracePeriod=30
Oct 30 23:34:13.826415 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:13.826379    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-kfmds container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 23:34:13.826828 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:13.826447    5664 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 23:34:16.707824 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.707798    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 23:34:16.802087 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802008    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-envoy\") pod \"7da38088-f4e1-4c68-ba94-b6a94434a021\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") "
Oct 30 23:34:16.802087 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802066    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-data\") pod \"7da38088-f4e1-4c68-ba94-b6a94434a021\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") "
Oct 30 23:34:16.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802115    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-kube-api-access-m698r\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-kube-api-access-m698r\") pod \"7da38088-f4e1-4c68-ba94-b6a94434a021\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") "
Oct 30 23:34:16.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802159    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-credential-socket\") pod \"7da38088-f4e1-4c68-ba94-b6a94434a021\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") "
Oct 30 23:34:16.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802248    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-podinfo\") pod \"7da38088-f4e1-4c68-ba94-b6a94434a021\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") "
Oct 30 23:34:16.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802282    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-certs\") pod \"7da38088-f4e1-4c68-ba94-b6a94434a021\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") "
Oct 30 23:34:16.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802306    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7da38088-f4e1-4c68-ba94-b6a94434a021-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da38088-f4e1-4c68-ba94-b6a94434a021-istiod-ca-cert\") pod \"7da38088-f4e1-4c68-ba94-b6a94434a021\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") "
Oct 30 23:34:16.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802338    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-token\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-token\") pod \"7da38088-f4e1-4c68-ba94-b6a94434a021\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") "
Oct 30 23:34:16.802314 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802373    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-socket\") pod \"7da38088-f4e1-4c68-ba94-b6a94434a021\" (UID: \"7da38088-f4e1-4c68-ba94-b6a94434a021\") "
Oct 30 23:34:16.802888 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802414    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-data" pod "7da38088-f4e1-4c68-ba94-b6a94434a021" (UID: "7da38088-f4e1-4c68-ba94-b6a94434a021"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.802888 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802527    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-certs" pod "7da38088-f4e1-4c68-ba94-b6a94434a021" (UID: "7da38088-f4e1-4c68-ba94-b6a94434a021"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.802888 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802760    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7da38088-f4e1-4c68-ba94-b6a94434a021-istiod-ca-cert" pod "7da38088-f4e1-4c68-ba94-b6a94434a021" (UID: "7da38088-f4e1-4c68-ba94-b6a94434a021"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:34:16.803229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802925    5664 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.803229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802951    5664 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da38088-f4e1-4c68-ba94-b6a94434a021-istiod-ca-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.803229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.802965    5664 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-data\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.803229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.803123    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-socket" pod "7da38088-f4e1-4c68-ba94-b6a94434a021" (UID: "7da38088-f4e1-4c68-ba94-b6a94434a021"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.803229 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.803146    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-credential-socket" pod "7da38088-f4e1-4c68-ba94-b6a94434a021" (UID: "7da38088-f4e1-4c68-ba94-b6a94434a021"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.804859 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.804835    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-envoy" pod "7da38088-f4e1-4c68-ba94-b6a94434a021" (UID: "7da38088-f4e1-4c68-ba94-b6a94434a021"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:34:16.805001 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.804916    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-kube-api-access-m698r" pod "7da38088-f4e1-4c68-ba94-b6a94434a021" (UID: "7da38088-f4e1-4c68-ba94-b6a94434a021"). InnerVolumeSpecName "kube-api-access-m698r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:16.805001 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.804977    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-podinfo" pod "7da38088-f4e1-4c68-ba94-b6a94434a021" (UID: "7da38088-f4e1-4c68-ba94-b6a94434a021"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:34:16.805101 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.804998    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-token" pod "7da38088-f4e1-4c68-ba94-b6a94434a021" (UID: "7da38088-f4e1-4c68-ba94-b6a94434a021"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:34:16.845162 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.845125    5664 generic.go:356] "Generic (PLEG): container finished" podID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerID="d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2" exitCode=0
Oct 30 23:34:16.845326 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.845176    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"
Oct 30 23:34:16.845326 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.845207    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" event={"ID":"7da38088-f4e1-4c68-ba94-b6a94434a021","Type":"ContainerDied","Data":"d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2"}
Oct 30 23:34:16.845326 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.845275    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds" event={"ID":"7da38088-f4e1-4c68-ba94-b6a94434a021","Type":"ContainerDied","Data":"a28d2baf569c74ea822aa6ef0b4547442dad443cfb322537cb8435ce93444f5d"}
Oct 30 23:34:16.845326 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.845305    5664 scope.go:122] "RemoveContainer" containerID="d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2"
Oct 30 23:34:16.853853 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.853837    5664 scope.go:122] "RemoveContainer" containerID="d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2"
Oct 30 23:34:16.854145 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:34:16.854126    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2\": container with ID starting with d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2 not found: ID does not exist" containerID="d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2"
Oct 30 23:34:16.854201 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.854155    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2"} err="failed to get container status \"d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2\": rpc error: code = NotFound desc = could not find container \"d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2\": container with ID starting with d5df5f182f5a7f3a2e692b96317bc354247d83508972c9e2d36e7a36720d62c2 not found: ID does not exist"
Oct 30 23:34:16.867863 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.867837    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"]
Oct 30 23:34:16.872322 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.872300    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-kfmds"]
Oct 30 23:34:16.904151 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.904126    5664 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-token\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.904151 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.904151    5664 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-workload-socket\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.904272 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.904162    5664 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-envoy\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.904272 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.904171    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m698r\" (UniqueName: \"kubernetes.io/projected/7da38088-f4e1-4c68-ba94-b6a94434a021-kube-api-access-m698r\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.904272 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.904180    5664 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da38088-f4e1-4c68-ba94-b6a94434a021-credential-socket\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:16.904272 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:16.904188    5664 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da38088-f4e1-4c68-ba94-b6a94434a021-istio-podinfo\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:34:17.867388 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:34:17.867350    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" path="/var/lib/kubelet/pods/7da38088-f4e1-4c68-ba94-b6a94434a021/volumes"
Oct 30 23:34:37.514485 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:34:37.514449    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:35:07.516576 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:35:07.516528    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:35:26.608283 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608243    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"]
Oct 30 23:35:26.608662 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608573    5664 cpu_manager.go:414] "containerMap: removing container" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy"
Oct 30 23:35:26.608662 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608585    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy"
Oct 30 23:35:26.608662 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608602    5664 cpu_manager.go:414] "containerMap: removing container" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main"
Oct 30 23:35:26.608662 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608607    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main"
Oct 30 23:35:26.608662 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608621    5664 cpu_manager.go:414] "containerMap: removing container" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="storage-initializer"
Oct 30 23:35:26.608662 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608626    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="storage-initializer"
Oct 30 23:35:26.608662 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608641    5664 cpu_manager.go:414] "containerMap: removing container" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="llm-d-routing-sidecar"
Oct 30 23:35:26.608662 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608645    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="llm-d-routing-sidecar"
Oct 30 23:35:26.608925 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608689    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="7da38088-f4e1-4c68-ba94-b6a94434a021" containerName="istio-proxy"
Oct 30 23:35:26.608925 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608697    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="main"
Oct 30 23:35:26.608925 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.608706    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="053dde5c-1ffd-47c4-8a1c-85c7187dc621" containerName="llm-d-routing-sidecar"
Oct 30 23:35:26.611585 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.611569    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.627201 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.627172    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"]
Oct 30 23:35:26.724658 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.724616    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.724658 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.724656    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrq9h\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-kube-api-access-hrq9h\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.724882 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.724741    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.724882 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.724777    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.724882 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.724801    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.724882 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.724851    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de06716f-b82c-4950-bcca-dc1d7ec51664-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.725102 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.724893    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.725102 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.724962    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.725102 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.724997    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.825797 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.825760    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826003 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.825807    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826003 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.825840    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826003 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.825855    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrq9h\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-kube-api-access-hrq9h\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826003 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.825875    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826003 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.825919    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826003 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.825944    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826299 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.826019    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de06716f-b82c-4950-bcca-dc1d7ec51664-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826299 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.826051    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826299 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.826212    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826299 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.826246    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-data\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826515 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.826354    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-credential-socket\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826640 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.826622    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-certs\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.826826 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.826808    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de06716f-b82c-4950-bcca-dc1d7ec51664-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.828434 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.828415    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-envoy\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.828482 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.828445    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-podinfo\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.839273 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.839247    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-token\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.839474 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.839452    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrq9h\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-kube-api-access-hrq9h\") pod \"openshift-ai-inference-openshift-default-86b669b864-8g5tm\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") " pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:26.924605 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:26.924576    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:27.203316 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:27.203232    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"]
Oct 30 23:35:27.206817 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:35:27.206792    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde06716f_b82c_4950_bcca_dc1d7ec51664.slice/crio-314f1b2ab0298918871869124d75011d2c65cf58e52d6cd1d4fa3892c4ca7190 WatchSource:0}: Error finding container 314f1b2ab0298918871869124d75011d2c65cf58e52d6cd1d4fa3892c4ca7190: Status 404 returned error can't find the container with id 314f1b2ab0298918871869124d75011d2c65cf58e52d6cd1d4fa3892c4ca7190
Oct 30 23:35:27.208778 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:27.208746    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:27.208870 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:27.208811    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:27.208870 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:27.208844    5664 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"}
Oct 30 23:35:28.137679 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:28.137643    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" event={"ID":"de06716f-b82c-4950-bcca-dc1d7ec51664","Type":"ContainerStarted","Data":"0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf"}
Oct 30 23:35:28.138124 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:28.137696    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" event={"ID":"de06716f-b82c-4950-bcca-dc1d7ec51664","Type":"ContainerStarted","Data":"314f1b2ab0298918871869124d75011d2c65cf58e52d6cd1d4fa3892c4ca7190"}
Oct 30 23:35:28.163797 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:28.163728    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" podStartSLOduration=2.163714092 podStartE2EDuration="2.163714092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:35:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:35:28.161690322 +0000 UTC m=+4558.413153788" watchObservedRunningTime="2026-10-30 23:35:28.163714092 +0000 UTC m=+4558.415177559"
Oct 30 23:35:28.925776 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:28.925734    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:28.927094 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:28.927072    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-8g5tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 23:35:28.927157 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:28.927122    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" podUID="de06716f-b82c-4950-bcca-dc1d7ec51664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 23:35:29.925269 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:29.925235    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-8g5tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 23:35:29.925665 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:29.925291    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" podUID="de06716f-b82c-4950-bcca-dc1d7ec51664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 23:35:30.925098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:30.925058    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-8g5tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 23:35:30.925317 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:30.925129    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" podUID="de06716f-b82c-4950-bcca-dc1d7ec51664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 23:35:31.926013 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:31.925973    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-8g5tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 23:35:31.926466 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:31.926046    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" podUID="de06716f-b82c-4950-bcca-dc1d7ec51664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 23:35:32.925722 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:32.925679    5664 patch_prober.go:29] interesting pod/openshift-ai-inference-openshift-default-86b669b864-8g5tm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Oct 30 23:35:32.925929 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:32.925747    5664 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" podUID="de06716f-b82c-4950-bcca-dc1d7ec51664" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Oct 30 23:35:33.928805 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:33.928774    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:33.929183 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:33.929086    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:33.929674 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:35:33.929659    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:35:37.515203 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:35:37.515159    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:36:07.515146 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:36:07.515107    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:36:26.495269 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.495190    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 30 23:36:26.498679 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.498656    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.504430 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.504403    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"openshift-service-ca.crt\""
Oct 30 23:36:26.504591 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.504446    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"kube-root-ca.crt\""
Oct 30 23:36:26.505496 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.505468    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:36:26.505626 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.505607    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"seaweedfs-s3-creds\""
Oct 30 23:36:26.505680 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.505634    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs\""
Oct 30 23:36:26.505715 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.505702    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"default-dockercfg-rmwcw\""
Oct 30 23:36:26.505784 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.505771    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisvc-model-fb-opt-125m-route-dc21cb-57cf7736-dockercfg-fjgh6\""
Oct 30 23:36:26.514460 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.514436    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 30 23:36:26.655238 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.655205    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.655238 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.655238    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.655488 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.655275    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/17a12f95-1673-409b-8b3f-57779225c3f8-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.655488 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.655409    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.655488 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.655457    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l7tn\" (UniqueName: \"kubernetes.io/projected/17a12f95-1673-409b-8b3f-57779225c3f8-kube-api-access-9l7tn\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.655614 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.655529    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.655614 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.655586    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.655720 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.655612    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17a12f95-1673-409b-8b3f-57779225c3f8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.756777 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.756676    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/17a12f95-1673-409b-8b3f-57779225c3f8-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.756777 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.756727    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.756777 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.756754    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l7tn\" (UniqueName: \"kubernetes.io/projected/17a12f95-1673-409b-8b3f-57779225c3f8-kube-api-access-9l7tn\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.756777 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.756777    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.757179 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.756796    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.757179 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.756811    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17a12f95-1673-409b-8b3f-57779225c3f8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.757179 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.756885    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.757179 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.756935    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.757390 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.757342    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.757432 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.757382    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.757477 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.757462    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.757517 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.757478    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/17a12f95-1673-409b-8b3f-57779225c3f8-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.757517 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.757488    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.759225 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.759202    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.759587 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.759565    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17a12f95-1673-409b-8b3f-57779225c3f8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.770503 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.770468    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l7tn\" (UniqueName: \"kubernetes.io/projected/17a12f95-1673-409b-8b3f-57779225c3f8-kube-api-access-9l7tn\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:26.816841 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:26.816811    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:27.110406 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:27.110320    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 30 23:36:27.113677 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:36:27.113648    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a12f95_1673_409b_8b3f_57779225c3f8.slice/crio-01c5309dd609d1a652d62f5e84f61d3c2a8540c1f355fe2a6b1cf4af72cca9ee WatchSource:0}: Error finding container 01c5309dd609d1a652d62f5e84f61d3c2a8540c1f355fe2a6b1cf4af72cca9ee: Status 404 returned error can't find the container with id 01c5309dd609d1a652d62f5e84f61d3c2a8540c1f355fe2a6b1cf4af72cca9ee
Oct 30 23:36:27.375430 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:27.375344    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"17a12f95-1673-409b-8b3f-57779225c3f8","Type":"ContainerStarted","Data":"3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08"}
Oct 30 23:36:27.375430 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:27.375402    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"17a12f95-1673-409b-8b3f-57779225c3f8","Type":"ContainerStarted","Data":"01c5309dd609d1a652d62f5e84f61d3c2a8540c1f355fe2a6b1cf4af72cca9ee"}
Oct 30 23:36:30.390138 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:30.390102    5664 generic.go:356] "Generic (PLEG): container finished" podID="17a12f95-1673-409b-8b3f-57779225c3f8" containerID="3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08" exitCode=0
Oct 30 23:36:30.390534 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:30.390174    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"17a12f95-1673-409b-8b3f-57779225c3f8","Type":"ContainerDied","Data":"3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08"}
Oct 30 23:36:31.397297 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:31.397262    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"17a12f95-1673-409b-8b3f-57779225c3f8","Type":"ContainerStarted","Data":"b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5"}
Oct 30 23:36:31.426295 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:31.426244    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podStartSLOduration=3.119779501 podStartE2EDuration="5.42623183s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.306452329s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:36:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:36:31.42430628 +0000 UTC m=+4621.675769782" watchObservedRunningTime="2026-10-30 23:36:31.42623183 +0000 UTC m=+4621.677695296"
Oct 30 23:36:31.816982 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:31.816891    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:36.817151 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:36.817112    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:36:36.818641 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:36.818592    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:36:37.515022 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:36:37.514980    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:36:46.817560 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:46.817516    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:36:48.795582 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.795542    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"]
Oct 30 23:36:48.799006 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.798988    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.806163 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.806136    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"odh-kserve-custom-ca-bundle\""
Oct 30 23:36:48.806163 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.806151    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"seaweedfs-s3-creds\""
Oct 30 23:36:48.806340 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.806176    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"kube-root-ca.crt\""
Oct 30 23:36:48.806340 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.806201    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"openshift-service-ca.crt\""
Oct 30 23:36:48.806340 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.806244    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"default-dockercfg-46nkk\""
Oct 30 23:36:48.806340 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.806311    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-prestop-hook-22727e61\"/\"prestop-hook-test-kserve-self-signed-certs\""
Oct 30 23:36:48.814574 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.814538    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"]
Oct 30 23:36:48.854757 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.854718    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-tmp-dir\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.854757 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.854759    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-kserve-provision-location\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.855058 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.854799    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-model-cache\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.855058 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.854939    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f864a0c3-ce4a-4956-b014-b5101ee9433b-tls-certs\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.855058 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.854971    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f864a0c3-ce4a-4956-b014-b5101ee9433b-cabundle-cert\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.855058 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.855003    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-dshm\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.855226 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.855092    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-home\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.855226 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.855147    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zm5l\" (UniqueName: \"kubernetes.io/projected/f864a0c3-ce4a-4956-b014-b5101ee9433b-kube-api-access-5zm5l\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.956392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.956351    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-home\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.956579 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.956412    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zm5l\" (UniqueName: \"kubernetes.io/projected/f864a0c3-ce4a-4956-b014-b5101ee9433b-kube-api-access-5zm5l\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.957188 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.956853    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-home\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.957188 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.956983    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-tmp-dir\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.957188 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.957100    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-kserve-provision-location\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.957572 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.957553    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-model-cache\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.957673 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.957656    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f864a0c3-ce4a-4956-b014-b5101ee9433b-tls-certs\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.957732 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.957712    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f864a0c3-ce4a-4956-b014-b5101ee9433b-cabundle-cert\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.957821 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.957801    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-dshm\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.957888 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.957820    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-kserve-provision-location\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.958027 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.958005    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-tmp-dir\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.960346 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.958974    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-model-cache\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.960346 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.959941    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f864a0c3-ce4a-4956-b014-b5101ee9433b-cabundle-cert\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.961163 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.961138    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-dshm\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.965427 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.961287    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f864a0c3-ce4a-4956-b014-b5101ee9433b-tls-certs\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:48.969915 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:48.969878    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zm5l\" (UniqueName: \"kubernetes.io/projected/f864a0c3-ce4a-4956-b014-b5101ee9433b-kube-api-access-5zm5l\") pod \"prestop-hook-test-kserve-5f8f955bf6-566bb\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") " pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:49.114274 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:49.114182    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:49.398136 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:49.398104    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"]
Oct 30 23:36:49.400933 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:36:49.400891    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf864a0c3_ce4a_4956_b014_b5101ee9433b.slice/crio-fe8612138d2448ca2eed4fc3c4483f2b4181843ffbe5c7ec054c28d24e449136 WatchSource:0}: Error finding container fe8612138d2448ca2eed4fc3c4483f2b4181843ffbe5c7ec054c28d24e449136: Status 404 returned error can't find the container with id fe8612138d2448ca2eed4fc3c4483f2b4181843ffbe5c7ec054c28d24e449136
Oct 30 23:36:49.475656 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:49.475612    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb" event={"ID":"f864a0c3-ce4a-4956-b014-b5101ee9433b","Type":"ContainerStarted","Data":"0c776c16c9dd9d0c68473c7c85d4a71ed45ec367514befb1d3723f1a58ddf946"}
Oct 30 23:36:49.475829 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:49.475684    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb" event={"ID":"f864a0c3-ce4a-4956-b014-b5101ee9433b","Type":"ContainerStarted","Data":"fe8612138d2448ca2eed4fc3c4483f2b4181843ffbe5c7ec054c28d24e449136"}
Oct 30 23:36:52.494493 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:52.494456    5664 generic.go:356] "Generic (PLEG): container finished" podID="f864a0c3-ce4a-4956-b014-b5101ee9433b" containerID="0c776c16c9dd9d0c68473c7c85d4a71ed45ec367514befb1d3723f1a58ddf946" exitCode=0
Oct 30 23:36:52.494985 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:52.494538    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb" event={"ID":"f864a0c3-ce4a-4956-b014-b5101ee9433b","Type":"ContainerDied","Data":"0c776c16c9dd9d0c68473c7c85d4a71ed45ec367514befb1d3723f1a58ddf946"}
Oct 30 23:36:53.502760 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:53.502719    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb" event={"ID":"f864a0c3-ce4a-4956-b014-b5101ee9433b","Type":"ContainerStarted","Data":"f8fb3af7860fb21ff59d8c66b05ad79f6416cf76c65a6fafa0fa4a469a3c5f17"}
Oct 30 23:36:53.530790 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:53.530727    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb" podStartSLOduration=3.104052399 podStartE2EDuration="5.530711049s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.42665865s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:36:53.528748671 +0000 UTC m=+4643.780212169" watchObservedRunningTime="2026-10-30 23:36:53.530711049 +0000 UTC m=+4643.782174549"
Oct 30 23:36:54.114447 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:54.114412    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:56.817802 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:56.817753    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:36:59.114650 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:59.114619    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:36:59.115952 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:36:59.115921    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb" podUID="f864a0c3-ce4a-4956-b014-b5101ee9433b" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": dial tcp 10.134.0.48:8000: connect: connection refused"
Oct 30 23:37:06.817844 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:37:06.817800    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:37:07.356355 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:37:07.356316    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"]
Oct 30 23:37:07.517155 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:37:07.517032    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:37:16.817779 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:37:16.817739    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:37:22.388988 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:37:22.388922    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb" podUID="f864a0c3-ce4a-4956-b014-b5101ee9433b" containerName="main" containerID="cri-o://f8fb3af7860fb21ff59d8c66b05ad79f6416cf76c65a6fafa0fa4a469a3c5f17" gracePeriod=45
Oct 30 23:37:26.817277 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:37:26.817233    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:37:36.818221 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:37:36.818168    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:37:37.518711 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:37:37.518668    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:37:46.818213 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:37:46.818125    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:37:56.817581 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:37:56.817537    5664 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:38:06.827098 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:06.827061    5664 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:38:06.834539 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:06.834511    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:38:07.417764 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:38:07.417732    5664 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf864a0c3_ce4a_4956_b014_b5101ee9433b.slice/crio-fe8612138d2448ca2eed4fc3c4483f2b4181843ffbe5c7ec054c28d24e449136\": RecentStats: unable to find data in memory cache]"
Oct 30 23:38:07.418025 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:38:07.417997    5664 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf864a0c3_ce4a_4956_b014_b5101ee9433b.slice/crio-fe8612138d2448ca2eed4fc3c4483f2b4181843ffbe5c7ec054c28d24e449136\": RecentStats: unable to find data in memory cache]"
Oct 30 23:38:07.517762 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:38:07.517722    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:38:07.848826 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:07.848798    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-prestop-hook-22727e61_prestop-hook-test-kserve-5f8f955bf6-566bb_f864a0c3-ce4a-4956-b014-b5101ee9433b/main/0.log"
Oct 30 23:38:07.849252 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:07.849168    5664 generic.go:356] "Generic (PLEG): container finished" podID="f864a0c3-ce4a-4956-b014-b5101ee9433b" containerID="f8fb3af7860fb21ff59d8c66b05ad79f6416cf76c65a6fafa0fa4a469a3c5f17" exitCode=137
Oct 30 23:38:07.849296 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:07.849245    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb" event={"ID":"f864a0c3-ce4a-4956-b014-b5101ee9433b","Type":"ContainerDied","Data":"f8fb3af7860fb21ff59d8c66b05ad79f6416cf76c65a6fafa0fa4a469a3c5f17"}
Oct 30 23:38:08.032664 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.032642    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-prestop-hook-22727e61_prestop-hook-test-kserve-5f8f955bf6-566bb_f864a0c3-ce4a-4956-b014-b5101ee9433b/main/0.log"
Oct 30 23:38:08.037955 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.037925    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:38:08.186951 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.186919    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-tmp-dir\") pod \"f864a0c3-ce4a-4956-b014-b5101ee9433b\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") "
Oct 30 23:38:08.187147 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.186965    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-kserve-provision-location\") pod \"f864a0c3-ce4a-4956-b014-b5101ee9433b\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") "
Oct 30 23:38:08.187147 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187002    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f864a0c3-ce4a-4956-b014-b5101ee9433b-kube-api-access-5zm5l\" (UniqueName: \"kubernetes.io/projected/f864a0c3-ce4a-4956-b014-b5101ee9433b-kube-api-access-5zm5l\") pod \"f864a0c3-ce4a-4956-b014-b5101ee9433b\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") "
Oct 30 23:38:08.187147 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187031    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-model-cache\") pod \"f864a0c3-ce4a-4956-b014-b5101ee9433b\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") "
Oct 30 23:38:08.187147 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187077    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f864a0c3-ce4a-4956-b014-b5101ee9433b-tls-certs\" (UniqueName: \"kubernetes.io/secret/f864a0c3-ce4a-4956-b014-b5101ee9433b-tls-certs\") pod \"f864a0c3-ce4a-4956-b014-b5101ee9433b\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") "
Oct 30 23:38:08.187147 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187106    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f864a0c3-ce4a-4956-b014-b5101ee9433b-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f864a0c3-ce4a-4956-b014-b5101ee9433b-cabundle-cert\") pod \"f864a0c3-ce4a-4956-b014-b5101ee9433b\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") "
Oct 30 23:38:08.187462 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187192    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-dshm\") pod \"f864a0c3-ce4a-4956-b014-b5101ee9433b\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") "
Oct 30 23:38:08.187462 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187232    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-home\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-home\") pod \"f864a0c3-ce4a-4956-b014-b5101ee9433b\" (UID: \"f864a0c3-ce4a-4956-b014-b5101ee9433b\") "
Oct 30 23:38:08.187462 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187363    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-model-cache" pod "f864a0c3-ce4a-4956-b014-b5101ee9433b" (UID: "f864a0c3-ce4a-4956-b014-b5101ee9433b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:08.187614 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187573    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:08.187984 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187952    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f864a0c3-ce4a-4956-b014-b5101ee9433b-cabundle-cert" pod "f864a0c3-ce4a-4956-b014-b5101ee9433b" (UID: "f864a0c3-ce4a-4956-b014-b5101ee9433b"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:38:08.188117 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.187986    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-home" pod "f864a0c3-ce4a-4956-b014-b5101ee9433b" (UID: "f864a0c3-ce4a-4956-b014-b5101ee9433b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:08.188117 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.188100    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-tmp-dir" pod "f864a0c3-ce4a-4956-b014-b5101ee9433b" (UID: "f864a0c3-ce4a-4956-b014-b5101ee9433b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:08.189518 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.189495    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f864a0c3-ce4a-4956-b014-b5101ee9433b-kube-api-access-5zm5l" pod "f864a0c3-ce4a-4956-b014-b5101ee9433b" (UID: "f864a0c3-ce4a-4956-b014-b5101ee9433b"). InnerVolumeSpecName "kube-api-access-5zm5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:38:08.190148 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.190121    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f864a0c3-ce4a-4956-b014-b5101ee9433b-tls-certs" pod "f864a0c3-ce4a-4956-b014-b5101ee9433b" (UID: "f864a0c3-ce4a-4956-b014-b5101ee9433b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:38:08.190148 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.190132    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-dshm" pod "f864a0c3-ce4a-4956-b014-b5101ee9433b" (UID: "f864a0c3-ce4a-4956-b014-b5101ee9433b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:08.208181 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.208153    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-kserve-provision-location" pod "f864a0c3-ce4a-4956-b014-b5101ee9433b" (UID: "f864a0c3-ce4a-4956-b014-b5101ee9433b"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:08.288742 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.288708    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f864a0c3-ce4a-4956-b014-b5101ee9433b-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:08.288742 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.288739    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/f864a0c3-ce4a-4956-b014-b5101ee9433b-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:08.288742 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.288750    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:08.288983 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.288758    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:08.288983 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.288787    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:08.288983 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.288797    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f864a0c3-ce4a-4956-b014-b5101ee9433b-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:08.288983 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.288807    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zm5l\" (UniqueName: \"kubernetes.io/projected/f864a0c3-ce4a-4956-b014-b5101ee9433b-kube-api-access-5zm5l\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:08.855033 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.855003    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/e2e-test-prestop-hook-22727e61_prestop-hook-test-kserve-5f8f955bf6-566bb_f864a0c3-ce4a-4956-b014-b5101ee9433b/main/0.log"
Oct 30 23:38:08.855484 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.855462    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb" event={"ID":"f864a0c3-ce4a-4956-b014-b5101ee9433b","Type":"ContainerDied","Data":"fe8612138d2448ca2eed4fc3c4483f2b4181843ffbe5c7ec054c28d24e449136"}
Oct 30 23:38:08.855536 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.855501    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"
Oct 30 23:38:08.855536 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.855515    5664 scope.go:122] "RemoveContainer" containerID="f8fb3af7860fb21ff59d8c66b05ad79f6416cf76c65a6fafa0fa4a469a3c5f17"
Oct 30 23:38:08.864996 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.864971    5664 scope.go:122] "RemoveContainer" containerID="0c776c16c9dd9d0c68473c7c85d4a71ed45ec367514befb1d3723f1a58ddf946"
Oct 30 23:38:08.880167 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.880140    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"]
Oct 30 23:38:08.886048 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:08.886026    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-prestop-hook-22727e61/prestop-hook-test-kserve-5f8f955bf6-566bb"]
Oct 30 23:38:09.868392 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:09.868355    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f864a0c3-ce4a-4956-b014-b5101ee9433b" path="/var/lib/kubelet/pods/f864a0c3-ce4a-4956-b014-b5101ee9433b/volumes"
Oct 30 23:38:12.399333 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:12.399301    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 30 23:38:27.423473 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:27.423433    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" containerID="cri-o://b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5" gracePeriod=45
Oct 30 23:38:27.828478 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:27.828391    5664 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": dial tcp 10.134.0.47:8000: connect: connection refused"
Oct 30 23:38:28.066530 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.066503    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:38:28.157788 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.157756    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-dshm\") pod \"17a12f95-1673-409b-8b3f-57779225c3f8\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") "
Oct 30 23:38:28.157969 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.157806    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/17a12f95-1673-409b-8b3f-57779225c3f8-tls-certs\" (UniqueName: \"kubernetes.io/secret/17a12f95-1673-409b-8b3f-57779225c3f8-tls-certs\") pod \"17a12f95-1673-409b-8b3f-57779225c3f8\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") "
Oct 30 23:38:28.157969 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.157833    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-tmp-dir\") pod \"17a12f95-1673-409b-8b3f-57779225c3f8\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") "
Oct 30 23:38:28.157969 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.157935    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17a12f95-1673-409b-8b3f-57779225c3f8-kube-api-access-9l7tn\" (UniqueName: \"kubernetes.io/projected/17a12f95-1673-409b-8b3f-57779225c3f8-kube-api-access-9l7tn\") pod \"17a12f95-1673-409b-8b3f-57779225c3f8\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") "
Oct 30 23:38:28.157969 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.157955    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-kserve-provision-location\") pod \"17a12f95-1673-409b-8b3f-57779225c3f8\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") "
Oct 30 23:38:28.158169 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.157977    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-model-cache\") pod \"17a12f95-1673-409b-8b3f-57779225c3f8\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") "
Oct 30 23:38:28.158169 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.157996    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17a12f95-1673-409b-8b3f-57779225c3f8-cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/17a12f95-1673-409b-8b3f-57779225c3f8-cabundle-cert\") pod \"17a12f95-1673-409b-8b3f-57779225c3f8\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") "
Oct 30 23:38:28.158169 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.158027    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-home\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-home\") pod \"17a12f95-1673-409b-8b3f-57779225c3f8\" (UID: \"17a12f95-1673-409b-8b3f-57779225c3f8\") "
Oct 30 23:38:28.158315 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.158277    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-model-cache" pod "17a12f95-1673-409b-8b3f-57779225c3f8" (UID: "17a12f95-1673-409b-8b3f-57779225c3f8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:28.158849 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.158821    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17a12f95-1673-409b-8b3f-57779225c3f8-cabundle-cert" pod "17a12f95-1673-409b-8b3f-57779225c3f8" (UID: "17a12f95-1673-409b-8b3f-57779225c3f8"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:38:28.158988 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.158850    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-home" pod "17a12f95-1673-409b-8b3f-57779225c3f8" (UID: "17a12f95-1673-409b-8b3f-57779225c3f8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:28.160403 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.160373    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17a12f95-1673-409b-8b3f-57779225c3f8-tls-certs" pod "17a12f95-1673-409b-8b3f-57779225c3f8" (UID: "17a12f95-1673-409b-8b3f-57779225c3f8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:38:28.160677 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.160653    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-dshm" pod "17a12f95-1673-409b-8b3f-57779225c3f8" (UID: "17a12f95-1673-409b-8b3f-57779225c3f8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:28.160769 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.160710    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17a12f95-1673-409b-8b3f-57779225c3f8-kube-api-access-9l7tn" pod "17a12f95-1673-409b-8b3f-57779225c3f8" (UID: "17a12f95-1673-409b-8b3f-57779225c3f8"). InnerVolumeSpecName "kube-api-access-9l7tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:38:28.169495 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.169472    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-tmp-dir" pod "17a12f95-1673-409b-8b3f-57779225c3f8" (UID: "17a12f95-1673-409b-8b3f-57779225c3f8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:28.186698 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.186668    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-kserve-provision-location" pod "17a12f95-1673-409b-8b3f-57779225c3f8" (UID: "17a12f95-1673-409b-8b3f-57779225c3f8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:38:28.259318 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.259284    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:28.259318 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.259313    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/17a12f95-1673-409b-8b3f-57779225c3f8-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:28.259318 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.259324    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:28.259536 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.259335    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l7tn\" (UniqueName: \"kubernetes.io/projected/17a12f95-1673-409b-8b3f-57779225c3f8-kube-api-access-9l7tn\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:28.259536 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.259346    5664 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-kserve-provision-location\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:28.259536 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.259355    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:28.259536 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.259363    5664 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/17a12f95-1673-409b-8b3f-57779225c3f8-cabundle-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:28.259536 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.259371    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/17a12f95-1673-409b-8b3f-57779225c3f8-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:38:28.945692 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.945650    5664 generic.go:356] "Generic (PLEG): container finished" podID="17a12f95-1673-409b-8b3f-57779225c3f8" containerID="b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5" exitCode=0
Oct 30 23:38:28.946156 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.945729    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"
Oct 30 23:38:28.946156 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.945744    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"17a12f95-1673-409b-8b3f-57779225c3f8","Type":"ContainerDied","Data":"b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5"}
Oct 30 23:38:28.946156 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.945803    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0" event={"ID":"17a12f95-1673-409b-8b3f-57779225c3f8","Type":"ContainerDied","Data":"01c5309dd609d1a652d62f5e84f61d3c2a8540c1f355fe2a6b1cf4af72cca9ee"}
Oct 30 23:38:28.946156 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.945826    5664 scope.go:122] "RemoveContainer" containerID="b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5"
Oct 30 23:38:28.954723 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.954705    5664 scope.go:122] "RemoveContainer" containerID="3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08"
Oct 30 23:38:28.962111 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.962094    5664 scope.go:122] "RemoveContainer" containerID="b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5"
Oct 30 23:38:28.962345 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:38:28.962326    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5\": container with ID starting with b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5 not found: ID does not exist" containerID="b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5"
Oct 30 23:38:28.962396 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.962354    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5"} err="failed to get container status \"b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5\": rpc error: code = NotFound desc = could not find container \"b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5\": container with ID starting with b11fd070ba2e74d379464e35d6f3f8ad988b9ea6aec130ece7fde01436da5db5 not found: ID does not exist"
Oct 30 23:38:28.962396 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.962380    5664 scope.go:122] "RemoveContainer" containerID="3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08"
Oct 30 23:38:28.962615 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:38:28.962581    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08\": container with ID starting with 3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08 not found: ID does not exist" containerID="3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08"
Oct 30 23:38:28.962652 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.962619    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08"} err="failed to get container status \"3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08\": rpc error: code = NotFound desc = could not find container \"3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08\": container with ID starting with 3bba3bb06cd7faa93aeb2647961c13917f344de5044b65b9bdc888698c13da08 not found: ID does not exist"
Oct 30 23:38:28.969297 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.969275    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 30 23:38:28.974193 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:28.974170    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0"]
Oct 30 23:38:29.867206 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:38:29.867174    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" path="/var/lib/kubelet/pods/17a12f95-1673-409b-8b3f-57779225c3f8/volumes"
Oct 30 23:38:37.515430 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:38:37.515385    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:39:07.516131 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:39:07.516091    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:39:37.515345 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:39:37.515309    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:39:42.809456 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809417    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"]
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809710    5664 cpu_manager.go:414] "containerMap: removing container" podUID="f864a0c3-ce4a-4956-b014-b5101ee9433b" containerName="storage-initializer"
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809720    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f864a0c3-ce4a-4956-b014-b5101ee9433b" containerName="storage-initializer"
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809739    5664 cpu_manager.go:414] "containerMap: removing container" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="storage-initializer"
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809745    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="storage-initializer"
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809756    5664 cpu_manager.go:414] "containerMap: removing container" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main"
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809761    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main"
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809779    5664 cpu_manager.go:414] "containerMap: removing container" podUID="f864a0c3-ce4a-4956-b014-b5101ee9433b" containerName="main"
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809784    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f864a0c3-ce4a-4956-b014-b5101ee9433b" containerName="main"
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809829    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="f864a0c3-ce4a-4956-b014-b5101ee9433b" containerName="main"
Oct 30 23:39:42.809874 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.809839    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="17a12f95-1673-409b-8b3f-57779225c3f8" containerName="main"
Oct 30 23:39:42.812742 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.812723    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:42.819303 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.819111    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"default-dockercfg-x4nkf\""
Oct 30 23:39:42.819303 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.819204    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"rolling-upgrade-test-kserve-self-signed-certs\""
Oct 30 23:39:42.819303 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.819266    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"openshift-service-ca.crt\""
Oct 30 23:39:42.819425 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.819339    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-rolling-upgrade-coordination-a9229638\"/\"kube-root-ca.crt\""
Oct 30 23:39:42.825560 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.825535    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"]
Oct 30 23:39:42.954042 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.954004    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-dshm\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:42.954216 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.954057    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5r9c\" (UniqueName: \"kubernetes.io/projected/5b7064d8-fc05-437e-bf84-250315ee0683-kube-api-access-z5r9c\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:42.954258 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.954209    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-model-cache\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:42.954302 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.954251    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b7064d8-fc05-437e-bf84-250315ee0683-tls-certs\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:42.954362 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.954341    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-home\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:42.954410 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:42.954391    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-tmp-dir\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.054812 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.054775    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5r9c\" (UniqueName: \"kubernetes.io/projected/5b7064d8-fc05-437e-bf84-250315ee0683-kube-api-access-z5r9c\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.054981 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.054835    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-model-cache\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.054981 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.054857    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b7064d8-fc05-437e-bf84-250315ee0683-tls-certs\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.054981 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.054878    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-home\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.055118 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.055032    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-tmp-dir\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.055118 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.055076    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-dshm\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.055260 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.055243    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-home\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.055333 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.055298    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-model-cache\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.055435 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.055413    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-tmp-dir\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.057344 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.057325    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-dshm\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.057580 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.057564    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b7064d8-fc05-437e-bf84-250315ee0683-tls-certs\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.068040 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.067981    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5r9c\" (UniqueName: \"kubernetes.io/projected/5b7064d8-fc05-437e-bf84-250315ee0683-kube-api-access-z5r9c\") pod \"rolling-upgrade-test-kserve-78f4d666cb-v48fz\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") " pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.125028 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.124992    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:43.398892 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:43.398855    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"]
Oct 30 23:39:43.402031 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:39:43.402006    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7064d8_fc05_437e_bf84_250315ee0683.slice/crio-6744c4c48758455af1f8516c7edfe61e1372a426e4b0559ceec8f5a796dff590 WatchSource:0}: Error finding container 6744c4c48758455af1f8516c7edfe61e1372a426e4b0559ceec8f5a796dff590: Status 404 returned error can't find the container with id 6744c4c48758455af1f8516c7edfe61e1372a426e4b0559ceec8f5a796dff590
Oct 30 23:39:44.276079 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:44.276044    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz" event={"ID":"5b7064d8-fc05-437e-bf84-250315ee0683","Type":"ContainerStarted","Data":"e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00"}
Oct 30 23:39:44.276472 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:44.276091    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz" event={"ID":"5b7064d8-fc05-437e-bf84-250315ee0683","Type":"ContainerStarted","Data":"6744c4c48758455af1f8516c7edfe61e1372a426e4b0559ceec8f5a796dff590"}
Oct 30 23:39:44.297141 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:44.297091    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz" podStartSLOduration=2.297077918 podStartE2EDuration="2.297077918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:39:44.295932128 +0000 UTC m=+4814.547395589" watchObservedRunningTime="2026-10-30 23:39:44.297077918 +0000 UTC m=+4814.548541383"
Oct 30 23:39:45.125967 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:45.125927    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:50.313679 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.313644    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"]
Oct 30 23:39:50.322718 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:39:50.322678    5664 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" containerID="e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00" cmd=["/bin/sleep","15"]
Oct 30 23:39:50.322861 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.322733    5664 handlers.go:85] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz" message="" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1"
Oct 30 23:39:50.322861 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:39:50.322746    5664 kuberuntime_container.go:780] "PreStop hook failed" err="rpc error: code = Unknown desc = command error: , stdout: , stderr: , exit code -1" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz" podUID="5b7064d8-fc05-437e-bf84-250315ee0683" containerName="main" containerID="cri-o://e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00"
Oct 30 23:39:50.322861 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.322796    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz" podUID="5b7064d8-fc05-437e-bf84-250315ee0683" containerName="main" containerID="cri-o://e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00" gracePeriod=60
Oct 30 23:39:50.561556 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.561531    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:50.721988 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.721957    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-tmp-dir\") pod \"5b7064d8-fc05-437e-bf84-250315ee0683\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") "
Oct 30 23:39:50.721988 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.721993    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-home\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-home\") pod \"5b7064d8-fc05-437e-bf84-250315ee0683\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") "
Oct 30 23:39:50.722250 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.722031    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-dshm\") pod \"5b7064d8-fc05-437e-bf84-250315ee0683\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") "
Oct 30 23:39:50.722250 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.722091    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5b7064d8-fc05-437e-bf84-250315ee0683-kube-api-access-z5r9c\" (UniqueName: \"kubernetes.io/projected/5b7064d8-fc05-437e-bf84-250315ee0683-kube-api-access-z5r9c\") pod \"5b7064d8-fc05-437e-bf84-250315ee0683\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") "
Oct 30 23:39:50.722250 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.722117    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5b7064d8-fc05-437e-bf84-250315ee0683-tls-certs\" (UniqueName: \"kubernetes.io/secret/5b7064d8-fc05-437e-bf84-250315ee0683-tls-certs\") pod \"5b7064d8-fc05-437e-bf84-250315ee0683\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") "
Oct 30 23:39:50.722250 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.722142    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-model-cache\") pod \"5b7064d8-fc05-437e-bf84-250315ee0683\" (UID: \"5b7064d8-fc05-437e-bf84-250315ee0683\") "
Oct 30 23:39:50.722459 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.722303    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-tmp-dir" pod "5b7064d8-fc05-437e-bf84-250315ee0683" (UID: "5b7064d8-fc05-437e-bf84-250315ee0683"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:39:50.722459 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.722320    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-home" pod "5b7064d8-fc05-437e-bf84-250315ee0683" (UID: "5b7064d8-fc05-437e-bf84-250315ee0683"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:39:50.722459 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.722446    5664 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-tmp-dir\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:39:50.722646 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.722466    5664 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-home\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:39:50.722646 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.722496    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-model-cache" pod "5b7064d8-fc05-437e-bf84-250315ee0683" (UID: "5b7064d8-fc05-437e-bf84-250315ee0683"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:39:50.724446 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.724427    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-dshm" pod "5b7064d8-fc05-437e-bf84-250315ee0683" (UID: "5b7064d8-fc05-437e-bf84-250315ee0683"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:39:50.724516 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.724469    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b7064d8-fc05-437e-bf84-250315ee0683-tls-certs" pod "5b7064d8-fc05-437e-bf84-250315ee0683" (UID: "5b7064d8-fc05-437e-bf84-250315ee0683"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 23:39:50.724578 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.724563    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b7064d8-fc05-437e-bf84-250315ee0683-kube-api-access-z5r9c" pod "5b7064d8-fc05-437e-bf84-250315ee0683" (UID: "5b7064d8-fc05-437e-bf84-250315ee0683"). InnerVolumeSpecName "kube-api-access-z5r9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:39:50.823146 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.823110    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5r9c\" (UniqueName: \"kubernetes.io/projected/5b7064d8-fc05-437e-bf84-250315ee0683-kube-api-access-z5r9c\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:39:50.823146 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.823138    5664 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b7064d8-fc05-437e-bf84-250315ee0683-tls-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:39:50.823146 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.823149    5664 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-model-cache\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:39:50.823380 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:50.823159    5664 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b7064d8-fc05-437e-bf84-250315ee0683-dshm\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:39:51.310530 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.310498    5664 generic.go:356] "Generic (PLEG): container finished" podID="5b7064d8-fc05-437e-bf84-250315ee0683" containerID="e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00" exitCode=0
Oct 30 23:39:51.310741 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.310542    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"
Oct 30 23:39:51.310741 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.310588    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz" event={"ID":"5b7064d8-fc05-437e-bf84-250315ee0683","Type":"ContainerDied","Data":"e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00"}
Oct 30 23:39:51.310741 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.310661    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz" event={"ID":"5b7064d8-fc05-437e-bf84-250315ee0683","Type":"ContainerDied","Data":"6744c4c48758455af1f8516c7edfe61e1372a426e4b0559ceec8f5a796dff590"}
Oct 30 23:39:51.310741 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.310685    5664 scope.go:122] "RemoveContainer" containerID="e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00"
Oct 30 23:39:51.319740 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.319712    5664 scope.go:122] "RemoveContainer" containerID="e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00"
Oct 30 23:39:51.320147 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:39:51.320021    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00\": container with ID starting with e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00 not found: ID does not exist" containerID="e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00"
Oct 30 23:39:51.320147 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.320046    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00"} err="failed to get container status \"e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00\": rpc error: code = NotFound desc = could not find container \"e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00\": container with ID starting with e824489be01c0d9b8c1b821e10e3cc4095e571909703312e6077f007c28abe00 not found: ID does not exist"
Oct 30 23:39:51.335627 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.335605    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"]
Oct 30 23:39:51.337804 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.337780    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-test-rolling-upgrade-coordination-a9229638/rolling-upgrade-test-kserve-78f4d666cb-v48fz"]
Oct 30 23:39:51.867812 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:39:51.867777    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5b7064d8-fc05-437e-bf84-250315ee0683" path="/var/lib/kubelet/pods/5b7064d8-fc05-437e-bf84-250315ee0683/volumes"
Oct 30 23:40:07.515074 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:40:07.515036    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:40:37.515756 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:40:37.515702    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:41:07.515008 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:41:07.514916    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:41:37.515061 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:41:37.515016    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:42:07.514347 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:42:07.514303    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:42:37.514661 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:42:37.514619    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:43:07.515644 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:43:07.515599    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:43:10.570594 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:10.570533    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-8g5tm_de06716f-b82c-4950-bcca-dc1d7ec51664/istio-proxy/0.log"
Oct 30 23:43:10.669182 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:10.669152    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-njf8w_fc55ee32-394e-44f8-b286-ae33c1f29b23/istio-proxy/0.log"
Oct 30 23:43:10.672478 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:10.672447    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-njf8w_fc55ee32-394e-44f8-b286-ae33c1f29b23/istio-proxy/1.log"
Oct 30 23:43:10.688706 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:10.688674    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-nn2ft_cf8b1942-8300-481c-9a58-e9afbeed93e5/istio-proxy/0.log"
Oct 30 23:43:10.692189 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:10.692166    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-nn2ft_cf8b1942-8300-481c-9a58-e9afbeed93e5/istio-proxy/1.log"
Oct 30 23:43:12.039816 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:12.039782    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-8g5tm_de06716f-b82c-4950-bcca-dc1d7ec51664/istio-proxy/0.log"
Oct 30 23:43:12.080214 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:12.080172    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-njf8w_fc55ee32-394e-44f8-b286-ae33c1f29b23/istio-proxy/0.log"
Oct 30 23:43:12.083833 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:12.083809    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-njf8w_fc55ee32-394e-44f8-b286-ae33c1f29b23/istio-proxy/1.log"
Oct 30 23:43:12.102685 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:12.102616    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-nn2ft_cf8b1942-8300-481c-9a58-e9afbeed93e5/istio-proxy/0.log"
Oct 30 23:43:12.106153 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:12.106130    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-nn2ft_cf8b1942-8300-481c-9a58-e9afbeed93e5/istio-proxy/1.log"
Oct 30 23:43:13.303706 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:13.303663    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-ml7lt_fed16662-e444-4db8-9be6-80fef392612e/limitador/0.log"
Oct 30 23:43:13.323948 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:13.323918    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-69b7bb8c5c-fsdcp_794a29cb-9298-4fd8-b2a8-ac2acdbffae8/manager/0.log"
Oct 30 23:43:16.027422 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.027379    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dn4ld/must-gather-6hmfk"]
Oct 30 23:43:16.027916 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.027766    5664 cpu_manager.go:414] "containerMap: removing container" podUID="5b7064d8-fc05-437e-bf84-250315ee0683" containerName="main"
Oct 30 23:43:16.027916 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.027781    5664 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5b7064d8-fc05-437e-bf84-250315ee0683" containerName="main"
Oct 30 23:43:16.027916 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.027842    5664 memory_manager.go:365] "RemoveStaleState removing state" podUID="5b7064d8-fc05-437e-bf84-250315ee0683" containerName="main"
Oct 30 23:43:16.030982 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.030965    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4ld/must-gather-6hmfk"
Oct 30 23:43:16.043433 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.043406    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4ld/must-gather-6hmfk"]
Oct 30 23:43:16.044217 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.044194    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dn4ld\"/\"kube-root-ca.crt\""
Oct 30 23:43:16.044452 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.044434    5664 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dn4ld\"/\"openshift-service-ca.crt\""
Oct 30 23:43:16.044518 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.044508    5664 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dn4ld\"/\"default-dockercfg-57457\""
Oct 30 23:43:16.150413 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.150377    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6rjx\" (UniqueName: \"kubernetes.io/projected/56e38023-e284-427c-8b0b-7531f09f2a29-kube-api-access-c6rjx\") pod \"must-gather-6hmfk\" (UID: \"56e38023-e284-427c-8b0b-7531f09f2a29\") " pod="openshift-must-gather-dn4ld/must-gather-6hmfk"
Oct 30 23:43:16.150413 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.150415    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/56e38023-e284-427c-8b0b-7531f09f2a29-must-gather-output\") pod \"must-gather-6hmfk\" (UID: \"56e38023-e284-427c-8b0b-7531f09f2a29\") " pod="openshift-must-gather-dn4ld/must-gather-6hmfk"
Oct 30 23:43:16.250853 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.250807    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6rjx\" (UniqueName: \"kubernetes.io/projected/56e38023-e284-427c-8b0b-7531f09f2a29-kube-api-access-c6rjx\") pod \"must-gather-6hmfk\" (UID: \"56e38023-e284-427c-8b0b-7531f09f2a29\") " pod="openshift-must-gather-dn4ld/must-gather-6hmfk"
Oct 30 23:43:16.250853 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.250847    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/56e38023-e284-427c-8b0b-7531f09f2a29-must-gather-output\") pod \"must-gather-6hmfk\" (UID: \"56e38023-e284-427c-8b0b-7531f09f2a29\") " pod="openshift-must-gather-dn4ld/must-gather-6hmfk"
Oct 30 23:43:16.251224 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.251205    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/56e38023-e284-427c-8b0b-7531f09f2a29-must-gather-output\") pod \"must-gather-6hmfk\" (UID: \"56e38023-e284-427c-8b0b-7531f09f2a29\") " pod="openshift-must-gather-dn4ld/must-gather-6hmfk"
Oct 30 23:43:16.263890 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.263861    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6rjx\" (UniqueName: \"kubernetes.io/projected/56e38023-e284-427c-8b0b-7531f09f2a29-kube-api-access-c6rjx\") pod \"must-gather-6hmfk\" (UID: \"56e38023-e284-427c-8b0b-7531f09f2a29\") " pod="openshift-must-gather-dn4ld/must-gather-6hmfk"
Oct 30 23:43:16.345020 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.344930    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4ld/must-gather-6hmfk"
Oct 30 23:43:16.636945 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.632470    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4ld/must-gather-6hmfk"]
Oct 30 23:43:16.638525 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:43:16.638488    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod56e38023_e284_427c_8b0b_7531f09f2a29.slice/crio-5f1d35314fdd8b31fa28ffc8c27c08ff7dacc79214100b99244700d7977473ca WatchSource:0}: Error finding container 5f1d35314fdd8b31fa28ffc8c27c08ff7dacc79214100b99244700d7977473ca: Status 404 returned error can't find the container with id 5f1d35314fdd8b31fa28ffc8c27c08ff7dacc79214100b99244700d7977473ca
Oct 30 23:43:16.640859 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:16.640840    5664 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 23:43:17.147447 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:17.147406    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4ld/must-gather-6hmfk" event={"ID":"56e38023-e284-427c-8b0b-7531f09f2a29","Type":"ContainerStarted","Data":"5f1d35314fdd8b31fa28ffc8c27c08ff7dacc79214100b99244700d7977473ca"}
Oct 30 23:43:18.154390 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:18.154353    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4ld/must-gather-6hmfk" event={"ID":"56e38023-e284-427c-8b0b-7531f09f2a29","Type":"ContainerStarted","Data":"578714bae2a3d4169ab4cee1889c0dc994207b78023ef4edf59062ea6f8f09f9"}
Oct 30 23:43:18.154810 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:18.154397    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4ld/must-gather-6hmfk" event={"ID":"56e38023-e284-427c-8b0b-7531f09f2a29","Type":"ContainerStarted","Data":"8e6e6e8675e95e2cc6e6bc188fd8628f94eb9a8ecdc43eb4a954b3b15adda831"}
Oct 30 23:43:18.175465 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:18.175410    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-dn4ld/must-gather-6hmfk" podStartSLOduration=1.327488782 podStartE2EDuration="2.175392537s" totalImagesPullingTime="847.903755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 23:43:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:43:18.174746354 +0000 UTC m=+5028.426209825" watchObservedRunningTime="2026-10-30 23:43:18.175392537 +0000 UTC m=+5028.426855998"
Oct 30 23:43:19.116509 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:19.116470    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bzxzm_1a882ca4-1b11-494c-a51e-fb3c93ef5606/global-pull-secret-syncer/0.log"
Oct 30 23:43:19.420135 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:19.420098    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ptpfs_f6695e08-7390-47f0-b5db-f474a466cf0a/konnectivity-agent/0.log"
Oct 30 23:43:19.538711 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:19.538672    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-192.us-east-2.compute.internal_320724d28f4bb4eb7790f5cce736ad83/haproxy/0.log"
Oct 30 23:43:37.515679 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:43:37.515638    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:43:52.720145 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:52.720064    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-ml7lt_fed16662-e444-4db8-9be6-80fef392612e/limitador/0.log"
Oct 30 23:43:52.771892 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:52.770844    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-69b7bb8c5c-fsdcp_794a29cb-9298-4fd8-b2a8-ac2acdbffae8/manager/0.log"
Oct 30 23:43:53.840039 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:53.840006    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-p4pkz_9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/kube-state-metrics/0.log"
Oct 30 23:43:53.862090 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:53.862063    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-p4pkz_9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/kube-rbac-proxy-main/0.log"
Oct 30 23:43:53.890255 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:53.890224    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-p4pkz_9b8cf69d-9ab1-4bfa-b927-509c0e847d1a/kube-rbac-proxy-self/0.log"
Oct 30 23:43:54.019928 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.019873    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2j5mw_77422c7f-1d2c-4ca8-968c-3e59beffbb1e/node-exporter/0.log"
Oct 30 23:43:54.062631 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.062603    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2j5mw_77422c7f-1d2c-4ca8-968c-3e59beffbb1e/kube-rbac-proxy/0.log"
Oct 30 23:43:54.091458 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.091387    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2j5mw_77422c7f-1d2c-4ca8-968c-3e59beffbb1e/init-textfile/0.log"
Oct 30 23:43:54.376199 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.376130    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-l6wf8_2fb7c085-6b91-409e-b15b-6d5b94f1c94c/kube-rbac-proxy-main/0.log"
Oct 30 23:43:54.414043 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.414014    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-l6wf8_2fb7c085-6b91-409e-b15b-6d5b94f1c94c/kube-rbac-proxy-self/0.log"
Oct 30 23:43:54.445789 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.445757    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-l6wf8_2fb7c085-6b91-409e-b15b-6d5b94f1c94c/openshift-state-metrics/0.log"
Oct 30 23:43:54.507298 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.507267    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2bedfe-3af0-427a-b3c2-3969f6660845/prometheus/0.log"
Oct 30 23:43:54.583964 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.583933    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2bedfe-3af0-427a-b3c2-3969f6660845/config-reloader/0.log"
Oct 30 23:43:54.660979 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.660949    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2bedfe-3af0-427a-b3c2-3969f6660845/thanos-sidecar/0.log"
Oct 30 23:43:54.707253 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.707223    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2bedfe-3af0-427a-b3c2-3969f6660845/kube-rbac-proxy-web/0.log"
Oct 30 23:43:54.746852 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.746820    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2bedfe-3af0-427a-b3c2-3969f6660845/kube-rbac-proxy/0.log"
Oct 30 23:43:54.783887 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.783856    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2bedfe-3af0-427a-b3c2-3969f6660845/kube-rbac-proxy-thanos/0.log"
Oct 30 23:43:54.815145 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:54.815117    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5a2bedfe-3af0-427a-b3c2-3969f6660845/init-config-reloader/0.log"
Oct 30 23:43:55.667727 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.667675    5664 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"]
Oct 30 23:43:55.675364 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.675334    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.702922 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.702405    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"]
Oct 30 23:43:55.763993 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.763957    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntcc2\" (UniqueName: \"kubernetes.io/projected/a343ca4d-6940-40c7-be2e-136c72c6d64c-kube-api-access-ntcc2\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.764165 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.764003    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-sys\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.764165 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.764077    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-proc\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.764165 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.764102    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-podres\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.764165 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.764128    5664 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-lib-modules\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.864693 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.864658    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-proc\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.864945 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.864715    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-podres\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.864945 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.864763    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-lib-modules\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.864945 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.864809    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntcc2\" (UniqueName: \"kubernetes.io/projected/a343ca4d-6940-40c7-be2e-136c72c6d64c-kube-api-access-ntcc2\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.864945 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.864815    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-proc\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.864945 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.864837    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-podres\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.864945 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.864855    5664 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-sys\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.865187 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.864959    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-sys\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.865187 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.864959    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a343ca4d-6940-40c7-be2e-136c72c6d64c-lib-modules\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.882485 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.882453    5664 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntcc2\" (UniqueName: \"kubernetes.io/projected/a343ca4d-6940-40c7-be2e-136c72c6d64c-kube-api-access-ntcc2\") pod \"perf-node-gather-daemonset-vmnxq\" (UID: \"a343ca4d-6940-40c7-be2e-136c72c6d64c\") " pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:55.995263 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:55.995182    5664 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:56.286289 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:56.286209    5664 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"]
Oct 30 23:43:56.290067 ip-10-0-132-192 kubenswrapper[5664]: W1030 23:43:56.290037    5664 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda343ca4d_6940_40c7_be2e_136c72c6d64c.slice/crio-46bfeb91cb6de572647566b75a97635564d3a4306590e3e5d0bce30f580feabc WatchSource:0}: Error finding container 46bfeb91cb6de572647566b75a97635564d3a4306590e3e5d0bce30f580feabc: Status 404 returned error can't find the container with id 46bfeb91cb6de572647566b75a97635564d3a4306590e3e5d0bce30f580feabc
Oct 30 23:43:56.335710 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:56.335663    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq" event={"ID":"a343ca4d-6940-40c7-be2e-136c72c6d64c","Type":"ContainerStarted","Data":"46bfeb91cb6de572647566b75a97635564d3a4306590e3e5d0bce30f580feabc"}
Oct 30 23:43:57.343465 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:57.343426    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq" event={"ID":"a343ca4d-6940-40c7-be2e-136c72c6d64c","Type":"ContainerStarted","Data":"594595975ddc34dbae1785d41424ed3c4d9babb7d945566728dab6c9af7c98fb"}
Oct 30 23:43:57.343947 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:57.343518    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:43:57.371181 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:57.371128    5664 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq" podStartSLOduration=2.371112928 podStartE2EDuration="2.371112928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 23:43:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 23:43:57.369189619 +0000 UTC m=+5067.620653087" watchObservedRunningTime="2026-10-30 23:43:57.371112928 +0000 UTC m=+5067.622576397"
Oct 30 23:43:59.257857 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:59.257827    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wghlv_7110fffb-3090-497a-a6d8-0260f3166ed1/dns/0.log"
Oct 30 23:43:59.285131 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:59.285106    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wghlv_7110fffb-3090-497a-a6d8-0260f3166ed1/kube-rbac-proxy/0.log"
Oct 30 23:43:59.314978 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:43:59.314943    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cf4tj_41e14309-abd7-407f-b98b-39005ec7d563/dns-node-resolver/0.log"
Oct 30 23:44:00.079256 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:00.079225    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kx7l5_777ef147-4f3f-4c89-ae2b-545aad6b31f1/node-ca/0.log"
Oct 30 23:44:01.484539 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:01.484488    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-8g5tm_de06716f-b82c-4950-bcca-dc1d7ec51664/istio-proxy/0.log"
Oct 30 23:44:01.555167 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:01.555132    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-njf8w_fc55ee32-394e-44f8-b286-ae33c1f29b23/istio-proxy/0.log"
Oct 30 23:44:01.558970 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:01.558946    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-njf8w_fc55ee32-394e-44f8-b286-ae33c1f29b23/istio-proxy/1.log"
Oct 30 23:44:01.589758 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:01.589732    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-nn2ft_cf8b1942-8300-481c-9a58-e9afbeed93e5/istio-proxy/0.log"
Oct 30 23:44:01.593308 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:01.593285    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-default-86b669b864-nn2ft_cf8b1942-8300-481c-9a58-e9afbeed93e5/istio-proxy/1.log"
Oct 30 23:44:02.548690 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:02.548663    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gt7k2_60e9435e-1582-493b-a73d-226b32c86abc/serve-healthcheck-canary/0.log"
Oct 30 23:44:03.351031 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:03.350998    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6w8dk_f9eaa777-0ec3-4c87-92a6-05e46f50066f/kube-rbac-proxy/0.log"
Oct 30 23:44:03.360063 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:03.360037    5664 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dn4ld/perf-node-gather-daemonset-vmnxq"
Oct 30 23:44:03.383154 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:03.383132    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6w8dk_f9eaa777-0ec3-4c87-92a6-05e46f50066f/exporter/0.log"
Oct 30 23:44:03.422070 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:03.422044    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6w8dk_f9eaa777-0ec3-4c87-92a6-05e46f50066f/extractor/0.log"
Oct 30 23:44:06.551205 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:06.551172    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6667976c75-8t7zf_7a321c9a-4218-45a7-a73a-83836840736b/manager/0.log"
Oct 30 23:44:06.648308 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:06.648277    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-8465994787-n5xg6_2c2b870c-91b9-4529-829c-08bd004cd85a/openshift-lws-operator/0.log"
Oct 30 23:44:07.525167 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:44:07.525135    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:44:15.232686 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:15.232651    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4bwqs_1f66a94c-5332-47dc-b3bb-9d7ba8ebe1b6/kube-multus/0.log"
Oct 30 23:44:15.547823 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:15.547754    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gbckc_72a3a151-21d2-4551-94b5-332f0b4f9542/kube-multus-additional-cni-plugins/0.log"
Oct 30 23:44:15.582556 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:15.582516    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gbckc_72a3a151-21d2-4551-94b5-332f0b4f9542/egress-router-binary-copy/0.log"
Oct 30 23:44:15.623208 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:15.623180    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gbckc_72a3a151-21d2-4551-94b5-332f0b4f9542/cni-plugins/0.log"
Oct 30 23:44:15.659512 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:15.659482    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gbckc_72a3a151-21d2-4551-94b5-332f0b4f9542/bond-cni-plugin/0.log"
Oct 30 23:44:15.698443 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:15.698417    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gbckc_72a3a151-21d2-4551-94b5-332f0b4f9542/routeoverride-cni/0.log"
Oct 30 23:44:15.747631 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:15.747586    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gbckc_72a3a151-21d2-4551-94b5-332f0b4f9542/whereabouts-cni-bincopy/0.log"
Oct 30 23:44:15.778543 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:15.778509    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gbckc_72a3a151-21d2-4551-94b5-332f0b4f9542/whereabouts-cni/0.log"
Oct 30 23:44:16.305152 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:16.305124    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cdmr4_7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa/network-metrics-daemon/0.log"
Oct 30 23:44:16.343096 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:16.343057    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cdmr4_7e6b9a4a-5287-4baf-a5e2-1de5f7b9befa/kube-rbac-proxy/0.log"
Oct 30 23:44:18.215652 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:18.215622    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdm75_76ab3954-a12f-484a-b32f-3de7517bc70f/ovn-controller/0.log"
Oct 30 23:44:18.254670 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:18.254634    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdm75_76ab3954-a12f-484a-b32f-3de7517bc70f/ovn-acl-logging/0.log"
Oct 30 23:44:18.289963 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:18.289928    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdm75_76ab3954-a12f-484a-b32f-3de7517bc70f/kube-rbac-proxy-node/0.log"
Oct 30 23:44:18.317918 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:18.317868    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdm75_76ab3954-a12f-484a-b32f-3de7517bc70f/kube-rbac-proxy-ovn-metrics/0.log"
Oct 30 23:44:18.356521 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:18.356491    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdm75_76ab3954-a12f-484a-b32f-3de7517bc70f/northd/0.log"
Oct 30 23:44:18.396310 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:18.396281    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdm75_76ab3954-a12f-484a-b32f-3de7517bc70f/nbdb/0.log"
Oct 30 23:44:18.438948 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:18.438893    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdm75_76ab3954-a12f-484a-b32f-3de7517bc70f/sbdb/0.log"
Oct 30 23:44:18.523996 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:18.523887    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hdm75_76ab3954-a12f-484a-b32f-3de7517bc70f/ovnkube-controller/0.log"
Oct 30 23:44:20.397506 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:20.397479    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-27nh7_3ea2d4d8-a3e0-4865-ae75-9db94e8c2d04/network-check-target-container/0.log"
Oct 30 23:44:22.050110 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:22.050078    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-h6x2s_bb6f24bb-ba37-400e-878f-267797c1903b/iptables-alerter/0.log"
Oct 30 23:44:23.248682 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:23.248650    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-ht6mr_dc0d5ce3-964a-4b73-9a51-ee2b4c2aa654/tuned/0.log"
Oct 30 23:44:27.918998 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:27.918965    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-849dk_d9093009-6c67-4d6b-8d3a-a388dbb078a3/csi-driver/0.log"
Oct 30 23:44:27.952705 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:27.952678    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-849dk_d9093009-6c67-4d6b-8d3a-a388dbb078a3/csi-node-driver-registrar/0.log"
Oct 30 23:44:27.985363 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:27.985335    5664 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-849dk_d9093009-6c67-4d6b-8d3a-a388dbb078a3/csi-liveness-probe/0.log"
Oct 30 23:44:37.514836 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:44:37.514799    5664 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6w8dk" containerName="extractor"
Oct 30 23:44:42.497063 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:42.497024    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"]
Oct 30 23:44:42.497947 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:42.497891    5664 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" podUID="de06716f-b82c-4950-bcca-dc1d7ec51664" containerName="istio-proxy" containerID="cri-o://0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf" gracePeriod=30
Oct 30 23:44:47.766160 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.766094    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:44:47.913761 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.913722    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-credential-socket\") pod \"de06716f-b82c-4950-bcca-dc1d7ec51664\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") "
Oct 30 23:44:47.913990 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.913825    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-data\") pod \"de06716f-b82c-4950-bcca-dc1d7ec51664\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") "
Oct 30 23:44:47.913990 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.913888    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-kube-api-access-hrq9h\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-kube-api-access-hrq9h\") pod \"de06716f-b82c-4950-bcca-dc1d7ec51664\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") "
Oct 30 23:44:47.913990 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.913936    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-certs\") pod \"de06716f-b82c-4950-bcca-dc1d7ec51664\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") "
Oct 30 23:44:47.914168 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914074    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-credential-socket" pod "de06716f-b82c-4950-bcca-dc1d7ec51664" (UID: "de06716f-b82c-4950-bcca-dc1d7ec51664"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.914168 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914106    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-data" pod "de06716f-b82c-4950-bcca-dc1d7ec51664" (UID: "de06716f-b82c-4950-bcca-dc1d7ec51664"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.914168 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914120    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-socket\") pod \"de06716f-b82c-4950-bcca-dc1d7ec51664\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") "
Oct 30 23:44:47.914333 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914190    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-token\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-token\") pod \"de06716f-b82c-4950-bcca-dc1d7ec51664\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") "
Oct 30 23:44:47.914333 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914242    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/de06716f-b82c-4950-bcca-dc1d7ec51664-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de06716f-b82c-4950-bcca-dc1d7ec51664-istiod-ca-cert\") pod \"de06716f-b82c-4950-bcca-dc1d7ec51664\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") "
Oct 30 23:44:47.914333 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914189    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-certs" pod "de06716f-b82c-4950-bcca-dc1d7ec51664" (UID: "de06716f-b82c-4950-bcca-dc1d7ec51664"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.914333 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914291    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-podinfo\") pod \"de06716f-b82c-4950-bcca-dc1d7ec51664\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") "
Oct 30 23:44:47.914333 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914320    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-socket" pod "de06716f-b82c-4950-bcca-dc1d7ec51664" (UID: "de06716f-b82c-4950-bcca-dc1d7ec51664"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.914333 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914323    5664 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-envoy\") pod \"de06716f-b82c-4950-bcca-dc1d7ec51664\" (UID: \"de06716f-b82c-4950-bcca-dc1d7ec51664\") "
Oct 30 23:44:47.914685 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914651    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de06716f-b82c-4950-bcca-dc1d7ec51664-istiod-ca-cert" pod "de06716f-b82c-4950-bcca-dc1d7ec51664" (UID: "de06716f-b82c-4950-bcca-dc1d7ec51664"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 23:44:47.914790 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914703    5664 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-data\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:47.914790 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914723    5664 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-certs\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:47.914790 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914739    5664 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-workload-socket\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:47.914790 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.914752    5664 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-credential-socket\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:47.916512 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.916485    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-token" pod "de06716f-b82c-4950-bcca-dc1d7ec51664" (UID: "de06716f-b82c-4950-bcca-dc1d7ec51664"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:47.916603 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.916566    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-kube-api-access-hrq9h" pod "de06716f-b82c-4950-bcca-dc1d7ec51664" (UID: "de06716f-b82c-4950-bcca-dc1d7ec51664"). InnerVolumeSpecName "kube-api-access-hrq9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 23:44:47.916974 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.916958    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-envoy" pod "de06716f-b82c-4950-bcca-dc1d7ec51664" (UID: "de06716f-b82c-4950-bcca-dc1d7ec51664"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 23:44:47.917256 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:47.917208    5664 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-podinfo" pod "de06716f-b82c-4950-bcca-dc1d7ec51664" (UID: "de06716f-b82c-4950-bcca-dc1d7ec51664"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 23:44:48.015493 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.015455    5664 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrq9h\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-kube-api-access-hrq9h\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.015493 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.015487    5664 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-token\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.015493 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.015497    5664 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/de06716f-b82c-4950-bcca-dc1d7ec51664-istiod-ca-cert\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.015749 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.015506    5664 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-podinfo\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.015749 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.015515    5664 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/de06716f-b82c-4950-bcca-dc1d7ec51664-istio-envoy\") on node \"ip-10-0-132-192.us-east-2.compute.internal\" DevicePath \"\""
Oct 30 23:44:48.570488 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.570453    5664 generic.go:356] "Generic (PLEG): container finished" podID="de06716f-b82c-4950-bcca-dc1d7ec51664" containerID="0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf" exitCode=0
Oct 30 23:44:48.570698 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.570515    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" event={"ID":"de06716f-b82c-4950-bcca-dc1d7ec51664","Type":"ContainerDied","Data":"0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf"}
Oct 30 23:44:48.570698 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.570520    5664 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"
Oct 30 23:44:48.570698 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.570565    5664 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm" event={"ID":"de06716f-b82c-4950-bcca-dc1d7ec51664","Type":"ContainerDied","Data":"314f1b2ab0298918871869124d75011d2c65cf58e52d6cd1d4fa3892c4ca7190"}
Oct 30 23:44:48.570698 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.570605    5664 scope.go:122] "RemoveContainer" containerID="0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf"
Oct 30 23:44:48.582501 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.580012    5664 scope.go:122] "RemoveContainer" containerID="0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf"
Oct 30 23:44:48.582501 ip-10-0-132-192 kubenswrapper[5664]: E1030 23:44:48.582300    5664 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf\": container with ID starting with 0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf not found: ID does not exist" containerID="0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf"
Oct 30 23:44:48.582501 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.582335    5664 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf"} err="failed to get container status \"0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf\": rpc error: code = NotFound desc = could not find container \"0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf\": container with ID starting with 0914f58d08bfc75d3745e59cd296d925902c0108cfee0e1d72eda34b302636bf not found: ID does not exist"
Oct 30 23:44:48.594351 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.594328    5664 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"]
Oct 30 23:44:48.601450 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:48.601429    5664 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-default-86b669b864-8g5tm"]
Oct 30 23:44:49.868039 ip-10-0-132-192 kubenswrapper[5664]: I1030 23:44:49.868003    5664 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="de06716f-b82c-4950-bcca-dc1d7ec51664" path="/var/lib/kubelet/pods/de06716f-b82c-4950-bcca-dc1d7ec51664/volumes"