Aug 05 17:10:24.569210 ip-10-0-141-246 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Aug 05 17:10:24.569226 ip-10-0-141-246 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Aug 05 17:10:24.569234 ip-10-0-141-246 systemd[1]: kubelet.service: Failed with result 'resources'.
Aug 05 17:10:24.569461 ip-10-0-141-246 systemd[1]: Failed to start Kubernetes Kubelet.
Aug 05 17:10:35.837508 ip-10-0-141-246 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Aug 05 17:10:35.837525 ip-10-0-141-246 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e4a37d6a3d0c47a59f35e5072bf563e3 --
Aug 05 17:12:34.057810 ip-10-0-141-246 systemd[1]: Starting Kubernetes Kubelet...
Aug 05 17:12:34.431162 ip-10-0-141-246 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:12:34.431162 ip-10-0-141-246 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Aug 05 17:12:34.431162 ip-10-0-141-246 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:12:34.431162 ip-10-0-141-246 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Aug 05 17:12:34.431162 ip-10-0-141-246 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Aug 05 17:12:34.432834 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.432729    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Aug 05 17:12:34.436407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436391    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:34.436407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436405    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:34.436407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436410    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436413    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436416    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436419    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436422    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436424    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436427    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436430    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436432    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436435    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436437    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436440    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436447    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436450    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436452    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436455    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436457    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436460    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436463    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436465    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:34.436531 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436468    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436472    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436474    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436477    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436479    2574 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436482    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436484    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436487    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436490    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436493    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436496    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436498    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436501    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436503    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436522    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436527    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436531    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436534    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436537    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436540    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:34.437032 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436542    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436545    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436548    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436551    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436554    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436557    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436559    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436562    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436564    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436567    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436570    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436572    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436574    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436577    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436580    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436582    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436585    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436587    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436590    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436592    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:34.437550 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436595    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436599    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436602    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436605    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436608    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436612    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436616    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436620    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436623    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436626    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436630    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436634    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436637    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436640    2574 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436643    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436645    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436649    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436652    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:34.438067 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436656    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436658    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436661    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436664    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436667    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436669    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436672    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436675    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436677    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436680    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436683    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436686    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436688    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436691    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436694    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436696    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436707    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436710    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.436713    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437429    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:34.438503 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437436    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437439    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437442    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437445    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437449    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437451    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437454    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437457    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437459    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437462    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437464    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437466    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437469    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437471    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437474    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437476    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437479    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437481    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437483    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437486    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:34.439013 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437488    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437491    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437493    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437496    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437498    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437501    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437503    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437524    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437530    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437533    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437536    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437538    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437543    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437547    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437550    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437554    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437557    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437560    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437563    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:34.439523 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437566    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437569    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437571    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437574    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437576    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437578    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437581    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437583    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437586    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437589    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437591    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437594    2574 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437596    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437598    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437601    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437604    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437607    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437610    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437612    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437615    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:34.439988 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437618    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437620    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437623    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437625    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437628    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437631    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437634    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437636    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437639    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437641    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437644    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437647    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437650    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437652    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437655    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437657    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437660    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437662    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437665    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437667    2574 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:34.440475 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437670    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437673    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437675    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437679    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437681    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437684    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437686    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437689    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437691    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437694    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437697    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437699    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437701    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437704    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437708    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437711    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437714    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437717    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.437720    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:34.440985 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438109    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438114    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438117    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438120    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438123    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438128    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438131    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438134    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438136    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438139    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438141    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438144    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438146    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438149    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438151    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438155    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438157    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438160    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438162    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:34.441440 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438165    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438168    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438171    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438173    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438176    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438178    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438181    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438184    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438186    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438189    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438191    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438195    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438199    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438202    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438204    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438207    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438210    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438213    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438218    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:34.441918 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438221    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438223    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438226    2574 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438229    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438232    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438234    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438237    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438240    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438243    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438246    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438249    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438251    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438254    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438257    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438260    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438263    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438266    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438268    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438270    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438273    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:34.442374 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438275    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438278    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438280    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438283    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438285    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438288    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438290    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438292    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438295    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438298    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438300    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438304    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438307    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438309    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438312    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438314    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438317    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438319    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438322    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438324    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:34.442934 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438327    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438329    2574 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438332    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438334    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438337    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438339    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438342    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438344    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438349    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438352    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438355    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438358    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438360    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438362    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438365    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438367    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438371    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438374    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438377    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438380    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:34.443419 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.438382    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438448    2574 flags.go:64] FLAG: --address="0.0.0.0"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438459    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438467    2574 flags.go:64] FLAG: --anonymous-auth="true"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438471    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438475    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438478    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438482    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438486    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438490    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438493    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438496    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438500    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438502    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438521    2574 flags.go:64] FLAG: --cgroup-root=""
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438526    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438530    2574 flags.go:64] FLAG: --client-ca-file=""
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438534    2574 flags.go:64] FLAG: --cloud-provider="external"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438537    2574 flags.go:64] FLAG: --cluster-dns="[]"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438542    2574 flags.go:64] FLAG: --cluster-domain=""
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438545    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438548    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438551    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438555    2574 flags.go:64] FLAG: --container-log-max-files="5"
Aug 05 17:12:34.443936 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438559    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438562    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438565    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438568    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438571    2574 flags.go:64] FLAG: --contention-profiling="false"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438574    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438577    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438580    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438583    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438587    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438590    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438594    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438597    2574 flags.go:64] FLAG: --enable-load-reader="false"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438600    2574 flags.go:64] FLAG: --enable-server="true"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438603    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438608    2574 flags.go:64] FLAG: --event-burst="100"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438611    2574 flags.go:64] FLAG: --event-qps="50"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438613    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438616    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438619    2574 flags.go:64] FLAG: --eviction-hard=""
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438623    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438626    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438629    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438632    2574 flags.go:64] FLAG: --eviction-soft=""
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438637    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Aug 05 17:12:34.444524 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438640    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438643    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438647    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438649    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438652    2574 flags.go:64] FLAG: --fail-swap-on="true"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438655    2574 flags.go:64] FLAG: --feature-gates=""
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438659    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438662    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438664    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438668    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438671    2574 flags.go:64] FLAG: --healthz-port="10248"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438674    2574 flags.go:64] FLAG: --help="false"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438677    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-141-246.ec2.internal"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438680    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438683    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438686    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438689    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438692    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438697    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438700    2574 flags.go:64] FLAG: --image-service-endpoint=""
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438703    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438706    2574 flags.go:64] FLAG: --kube-api-burst="100"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438709    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438712    2574 flags.go:64] FLAG: --kube-api-qps="50"
Aug 05 17:12:34.445151 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438715    2574 flags.go:64] FLAG: --kube-reserved=""
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438718    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438721    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438724    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438727    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438730    2574 flags.go:64] FLAG: --lock-file=""
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438733    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438737    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438740    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438745    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438748    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438751    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438754    2574 flags.go:64] FLAG: --logging-format="text"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438756    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438760    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438762    2574 flags.go:64] FLAG: --manifest-url=""
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438765    2574 flags.go:64] FLAG: --manifest-url-header=""
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438769    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438772    2574 flags.go:64] FLAG: --max-open-files="1000000"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438776    2574 flags.go:64] FLAG: --max-pods="110"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438779    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438782    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438785    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438788    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438791    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Aug 05 17:12:34.445753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438794    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438798    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438804    2574 flags.go:64] FLAG: --node-status-max-images="50"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438807    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438810    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438813    2574 flags.go:64] FLAG: --pod-cidr=""
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438817    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:267c8c33005f5dc3d4121af550b1c396e30e2928a13624d7362adfbfb7f440f8"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438822    2574 flags.go:64] FLAG: --pod-manifest-path=""
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438825    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438828    2574 flags.go:64] FLAG: --pods-per-core="0"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438831    2574 flags.go:64] FLAG: --port="10250"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438834    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438837    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e5ca6e8ff3ade84b"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438840    2574 flags.go:64] FLAG: --qos-reserved=""
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438842    2574 flags.go:64] FLAG: --read-only-port="10255"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438846    2574 flags.go:64] FLAG: --register-node="true"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438849    2574 flags.go:64] FLAG: --register-with-taints=""
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438853    2574 flags.go:64] FLAG: --registry-burst="10"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438856    2574 flags.go:64] FLAG: --registry-qps="5"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438859    2574 flags.go:64] FLAG: --reserved-cpus=""
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438861    2574 flags.go:64] FLAG: --reserved-memory=""
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438865    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438868    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438871    2574 flags.go:64] FLAG: --rotate-certificates="false"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438874    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Aug 05 17:12:34.446390 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438877    2574 flags.go:64] FLAG: --runonce="false"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438879    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438882    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438885    2574 flags.go:64] FLAG: --seccomp-default="false"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438888    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438891    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438894    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438897    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438901    2574 flags.go:64] FLAG: --storage-driver-password="root"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438904    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438907    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438909    2574 flags.go:64] FLAG: --storage-driver-user="root"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438912    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438915    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438919    2574 flags.go:64] FLAG: --system-cgroups=""
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438922    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438927    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438930    2574 flags.go:64] FLAG: --tls-cert-file=""
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438933    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438937    2574 flags.go:64] FLAG: --tls-min-version=""
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438940    2574 flags.go:64] FLAG: --tls-private-key-file=""
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438943    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438947    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438950    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438953    2574 flags.go:64] FLAG: --v="2"
Aug 05 17:12:34.447027 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438957    2574 flags.go:64] FLAG: --version="false"
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438961    2574 flags.go:64] FLAG: --vmodule=""
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438964    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.438968    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439078    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439081    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439088    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439091    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439093    2574 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439096    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439099    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439102    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439104    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439107    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439109    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439113    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439116    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439119    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439121    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439124    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439127    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:34.447645 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439129    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439132    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439135    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439138    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439140    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439143    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439146    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439148    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439152    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439155    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439157    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439160    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439162    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439165    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439167    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439170    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439172    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439175    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439177    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439180    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:34.448166 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439183    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439185    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439188    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439190    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439193    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439195    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439199    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439201    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439204    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439206    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439210    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439214    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439217    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439220    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439223    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439226    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439229    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439233    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439237    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:34.448672 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439241    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439245    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439248    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439250    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439253    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439256    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439259    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439262    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439264    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439267    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439270    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439272    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439275    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439277    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439280    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439283    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439285    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439288    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439291    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439294    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:34.449163 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439297    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439299    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439302    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439304    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439307    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439309    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439312    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439315    2574 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439317    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439320    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439322    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439325    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439328    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439332    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439334    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439337    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439339    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439342    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439344    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439347    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:34.449661 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439349    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439352    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.439355    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.440023    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.446793    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.446806    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.447681    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.447690    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447747    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447752    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447755    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447759    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447763    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447766    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447768    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:34.450258 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447771    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447774    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447777    2574 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447779    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447782    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447785    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447788    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447791    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447793    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447796    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447799    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447802    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447805    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447808    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447810    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447813    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447816    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447818    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447821    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447823    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:34.450895 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447826    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447829    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447832    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447835    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447838    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447841    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447843    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447846    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447849    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447852    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447854    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447858    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447861    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447865    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447869    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447872    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447875    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447878    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447882    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:34.451436 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447884    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447887    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447890    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447893    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447895    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447898    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447900    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447903    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447905    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447908    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447910    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447913    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447916    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447918    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447921    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447925    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447928    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447931    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447933    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:34.451914 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447936    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447939    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447941    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447944    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447946    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447949    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447951    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447954    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447957    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447960    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447962    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447965    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447967    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447970    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447973    2574 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447975    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447978    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447981    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447983    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447986    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:34.452407 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447988    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447991    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447993    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.447997    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448001    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448003    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448006    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448009    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448012    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448015    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448017    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448020    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448022    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448025    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.448030    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:12:34.452931 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448157    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448163    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448166    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448169    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448172    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448175    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448178    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448181    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448184    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448187    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448189    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448193    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448195    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448198    2574 feature_gate.go:328] unrecognized feature gate: Example2
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448200    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448203    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448205    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448208    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448210    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Aug 05 17:12:34.453310 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448213    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448215    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448219    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448222    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448225    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448229    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448233    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448236    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448239    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448241    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448244    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448246    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448249    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448251    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448254    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448256    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448259    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448261    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448263    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Aug 05 17:12:34.453793 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448266    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448268    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448271    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448273    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448276    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448278    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448281    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448283    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448286    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448288    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448291    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448293    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448296    2574 feature_gate.go:328] unrecognized feature gate: Example
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448298    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448300    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448303    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448305    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448308    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448310    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448313    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Aug 05 17:12:34.454254 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448316    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448318    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448321    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448323    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448326    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448330    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448333    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448336    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448339    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448342    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448344    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448347    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448350    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448352    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448356    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448360    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448362    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448365    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448367    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Aug 05 17:12:34.454752 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448370    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448373    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448376    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448378    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448381    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448383    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448386    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448388    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448391    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448393    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448396    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448398    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448401    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448404    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448407    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448409    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448412    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448414    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448417    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448419    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Aug 05 17:12:34.455201 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448422    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Aug 05 17:12:34.455795 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:34.448424    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Aug 05 17:12:34.455795 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.448429    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Aug 05 17:12:34.455795 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.448534    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Aug 05 17:12:34.455795 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.452030    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Aug 05 17:12:34.455795 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.453563    2574 server.go:1019] "Starting client certificate rotation"
Aug 05 17:12:34.455795 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.453665    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:12:34.455795 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.453698    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Aug 05 17:12:34.457114 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.457100    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:12:34.458267 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.458248    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Aug 05 17:12:34.469925 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.469908    2574 log.go:25] "Validated CRI v1 runtime API"
Aug 05 17:12:34.472562 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.472539    2574 log.go:25] "Validated CRI v1 image API"
Aug 05 17:12:34.473689 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.473676    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Aug 05 17:12:34.476100 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.476071    2574 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 815cc914-9cc1-4aa9-8ef3-8a1bbb4c0ef1:/dev/nvme0n1p4 dc05ddbc-421b-492d-84ee-633b5ea87147:/dev/nvme0n1p3]
Aug 05 17:12:34.476100 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.476088    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Aug 05 17:12:34.482402 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.482288    2574 manager.go:216] Machine: {Timestamp:2026-08-05 17:12:34.48107469 +0000 UTC m=+0.312835493 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100146 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec266373d6b0d79dc1200d6180a88f4d SystemUUID:ec266373-d6b0-d79d-c120-0d6180a88f4d BootID:e4a37d6a-3d0c-47a5-9f35-e5072bf563e3 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ec:3a:56:bf:fd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ec:3a:56:bf:fd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:ec:de:c1:ef:d1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Aug 05 17:12:34.482402 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.482399    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Aug 05 17:12:34.482578 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.482466    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.131.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260727-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Aug 05 17:12:34.482663 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.482647    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:12:34.485273 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.485246    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Aug 05 17:12:34.485411 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.485274    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-246.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Aug 05 17:12:34.486068 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.486058    2574 topology_manager.go:138] "Creating topology manager with none policy"
Aug 05 17:12:34.486101 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.486071    2574 container_manager_linux.go:309] "Creating device plugin manager"
Aug 05 17:12:34.486101 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.486083    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:12:34.486702 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.486693    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Aug 05 17:12:34.487698 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.487688    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Aug 05 17:12:34.491470 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.491459    2574 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:12:34.491600 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.491590    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Aug 05 17:12:34.492216 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.492204    2574 kubelet.go:486] "Attempting to sync node with API server"
Aug 05 17:12:34.492277 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.492232    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Aug 05 17:12:34.492277 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.492248    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Aug 05 17:12:34.492277 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.492257    2574 kubelet.go:398] "Adding apiserver pod source"
Aug 05 17:12:34.492277 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.492264    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Aug 05 17:12:34.492694 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.492679    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:12:34.492772 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.492706    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Aug 05 17:12:34.495871 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.495827    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Aug 05 17:12:34.497422 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.497407    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Aug 05 17:12:34.497492 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.497438    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Aug 05 17:12:34.498424 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498411    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Aug 05 17:12:34.498468 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498433    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Aug 05 17:12:34.498468 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498442    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Aug 05 17:12:34.498468 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498448    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Aug 05 17:12:34.498468 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498458    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Aug 05 17:12:34.498468 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498463    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Aug 05 17:12:34.498627 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498471    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Aug 05 17:12:34.498627 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498476    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Aug 05 17:12:34.498627 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498483    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Aug 05 17:12:34.498627 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498488    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Aug 05 17:12:34.498627 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498496    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Aug 05 17:12:34.498627 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.498501    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Aug 05 17:12:34.499092 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.499084    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Aug 05 17:12:34.499092 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.499092    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Aug 05 17:12:34.502283 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.502268    2574 server.go:1268] "Started kubelet"
Aug 05 17:12:34.502367 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.502339    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Aug 05 17:12:34.502467 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.502412    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Aug 05 17:12:34.502556 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.502492    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Aug 05 17:12:34.503101 ip-10-0-141-246 systemd[1]: Started Kubernetes Kubelet.
Aug 05 17:12:34.504037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.503915    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-246.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Aug 05 17:12:34.504037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.504027    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Aug 05 17:12:34.504366 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.504336    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-246.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Aug 05 17:12:34.504366 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.504336    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Aug 05 17:12:34.504961 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.504945    2574 server.go:310] "Adding debug handlers to kubelet server"
Aug 05 17:12:34.506334 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.506314    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bq4wj"
Aug 05 17:12:34.507362 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.507344    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Aug 05 17:12:34.508619 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.507127    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-246.ec2.internal.18c8f80570bb3333  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-246.ec2.internal,UID:ip-10-0-141-246.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-246.ec2.internal,},FirstTimestamp:2026-08-05 17:12:34.502243123 +0000 UTC m=+0.334003924,LastTimestamp:2026-08-05 17:12:34.502243123 +0000 UTC m=+0.334003924,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-246.ec2.internal,}"
Aug 05 17:12:34.510403 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.510388    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Aug 05 17:12:34.510403 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.510397    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Aug 05 17:12:34.510542 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.510526    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:34.511160 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511075    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Aug 05 17:12:34.511160 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511096    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Aug 05 17:12:34.511160 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511139    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Aug 05 17:12:34.511355 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511222    2574 reconstruct.go:97] "Volume reconstruction finished"
Aug 05 17:12:34.511355 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511232    2574 reconciler.go:29] "Reconciler: start to sync state"
Aug 05 17:12:34.511355 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511282    2574 factory.go:55] Registering systemd factory
Aug 05 17:12:34.511355 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511297    2574 factory.go:223] Registration of the systemd container factory successfully
Aug 05 17:12:34.511533 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511480    2574 factory.go:153] Registering CRI-O factory
Aug 05 17:12:34.511533 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511493    2574 factory.go:223] Registration of the crio container factory successfully
Aug 05 17:12:34.511636 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511566    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Aug 05 17:12:34.511636 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511587    2574 factory.go:103] Registering Raw factory
Aug 05 17:12:34.511636 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511597    2574 manager.go:1196] Started watching for new ooms in manager
Aug 05 17:12:34.512003 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.511990    2574 manager.go:349] Starting recovery of all containers
Aug 05 17:12:34.512787 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.512768    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Aug 05 17:12:34.512906 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.512887    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-246.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Aug 05 17:12:34.518305 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.518283    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bq4wj"
Aug 05 17:12:34.524231 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.524213    2574 manager.go:354] Recovery completed
Aug 05 17:12:34.528526 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.528501    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:34.530105 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.530086    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:34.530192 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.530119    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:34.530192 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.530134    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:34.530676 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.530653    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Aug 05 17:12:34.530676 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.530667    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Aug 05 17:12:34.530676 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.530668    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:34.530805 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.530685    2574 state_mem.go:36] "Initialized new in-memory state store"
Aug 05 17:12:34.531484 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.531471    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:34.531572 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.531494    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:34.531572 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.531503    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:34.531781 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.531770    2574 policy_none.go:49] "None policy: Start"
Aug 05 17:12:34.531814 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.531786    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Aug 05 17:12:34.531814 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.531796    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Aug 05 17:12:34.532479 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.532419    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-246.ec2.internal.18c8f8057264509f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-246.ec2.internal,UID:ip-10-0-141-246.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-246.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-246.ec2.internal,},FirstTimestamp:2026-08-05 17:12:34.530103455 +0000 UTC m=+0.361864263,LastTimestamp:2026-08-05 17:12:34.530103455 +0000 UTC m=+0.361864263,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-246.ec2.internal,}"
Aug 05 17:12:34.532720 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.532696    2574 policy_none.go:47] "Start"
Aug 05 17:12:34.573603 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.573465    2574 manager.go:337] "Starting Device Plugin manager"
Aug 05 17:12:34.573706 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.573637    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Aug 05 17:12:34.573706 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.573650    2574 server.go:87] "Starting device plugin registration server"
Aug 05 17:12:34.573835 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.573824    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Aug 05 17:12:34.573882 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.573837    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Aug 05 17:12:34.573942 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.573923    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Aug 05 17:12:34.574024 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.574008    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Aug 05 17:12:34.574024 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.574021    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Aug 05 17:12:34.574495 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.574473    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Aug 05 17:12:34.574580 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.574549    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:34.674909 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.674887    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:34.675956 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.675729    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:34.675956 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.675753    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:34.675956 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.675763    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:34.675956 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.675788    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-246.ec2.internal"
Aug 05 17:12:34.682587 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.682542    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-246.ec2.internal"
Aug 05 17:12:34.682587 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.682573    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-246.ec2.internal\": node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:34.694901 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.694880    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:34.715042 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.715006    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Aug 05 17:12:34.716179 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.716162    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Aug 05 17:12:34.716265 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.716195    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Aug 05 17:12:34.716265 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.716219    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Aug 05 17:12:34.716265 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:34.716248    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Aug 05 17:12:34.718924 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:34.718910    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:35.928471 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:35.928441    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:35.949829 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.949807    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal"]
Aug 05 17:12:35.949903 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.949873    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:35.950713 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.950695    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:35.950783 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.950724    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:35.950783 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.950734    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:35.951973 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.951962    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:35.952124 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.952099    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:35.952173 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.952127    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:35.952645 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.952632    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:35.952684 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.952657    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:35.952684 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.952670    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:35.952684 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.952681    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:35.952777 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.952698    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:35.952777 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.952707    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:35.953839 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.953826    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:35.953886 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.953850    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Aug 05 17:12:35.954480 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.954466    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientMemory"
Aug 05 17:12:35.954575 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.954490    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasNoDiskPressure"
Aug 05 17:12:35.954575 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:35.954501    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeHasSufficientPID"
Aug 05 17:12:35.993574 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:35.993558    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-246.ec2.internal\" not found" node="ip-10-0-141-246.ec2.internal"
Aug 05 17:12:35.997713 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:35.997695    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-246.ec2.internal\" not found" node="ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.029368 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:36.029349    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:36.045466 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.045446    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/80068e53ca81b677a6b8796d1e4de03a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal\" (UID: \"80068e53ca81b677a6b8796d1e4de03a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.045548 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.045468    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80068e53ca81b677a6b8796d1e4de03a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal\" (UID: \"80068e53ca81b677a6b8796d1e4de03a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.045548 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.045485    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ea1cdff287318a9cf0bdf169156a32c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-246.ec2.internal\" (UID: \"0ea1cdff287318a9cf0bdf169156a32c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.129907 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:36.129829    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:36.146166 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.146147    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ea1cdff287318a9cf0bdf169156a32c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-246.ec2.internal\" (UID: \"0ea1cdff287318a9cf0bdf169156a32c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.146249 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.146171    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/80068e53ca81b677a6b8796d1e4de03a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal\" (UID: \"80068e53ca81b677a6b8796d1e4de03a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.146249 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.146198    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80068e53ca81b677a6b8796d1e4de03a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal\" (UID: \"80068e53ca81b677a6b8796d1e4de03a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.146249 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.146224    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80068e53ca81b677a6b8796d1e4de03a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal\" (UID: \"80068e53ca81b677a6b8796d1e4de03a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.146361 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.146250    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/80068e53ca81b677a6b8796d1e4de03a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal\" (UID: \"80068e53ca81b677a6b8796d1e4de03a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.146361 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.146251    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ea1cdff287318a9cf0bdf169156a32c-config\") pod \"kube-apiserver-proxy-ip-10-0-141-246.ec2.internal\" (UID: \"0ea1cdff287318a9cf0bdf169156a32c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.230334 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:36.230307    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:36.297851 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.297823    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.301111 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.301093    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.330433 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:36.330410    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:36.430951 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:36.430899    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:36.465229 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.465202    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:36.513186 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.513165    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:36.531238 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:36.531221    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:36.585778 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.585758    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Aug 05 17:12:36.586198 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.585906    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 05 17:12:36.586198 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.585919    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 05 17:12:36.586198 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.585921    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Aug 05 17:12:36.632211 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:36.632191    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-246.ec2.internal\" not found"
Aug 05 17:12:36.657578 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.657547    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-08-04 17:07:35 +0000 UTC" deadline="2028-03-25 04:49:51.980590541 +0000 UTC"
Aug 05 17:12:36.657578 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.657570    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14339h37m15.323023126s"
Aug 05 17:12:36.717049 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.716980    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:36.743708 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.743685    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.756081 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.756064    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:12:36.757330 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.757318    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal"
Aug 05 17:12:36.766122 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.766105    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Aug 05 17:12:36.900867 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:36.900841    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80068e53ca81b677a6b8796d1e4de03a.slice/crio-28187a12c2267b5a380f675281f6c67e6e8a2a4455fbc382089b5a335d3db600 WatchSource:0}: Error finding container 28187a12c2267b5a380f675281f6c67e6e8a2a4455fbc382089b5a335d3db600: Status 404 returned error can't find the container with id 28187a12c2267b5a380f675281f6c67e6e8a2a4455fbc382089b5a335d3db600
Aug 05 17:12:36.904455 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:36.904436    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:12:36.990225 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:36.990203    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea1cdff287318a9cf0bdf169156a32c.slice/crio-66b4253303c2a91c231c2b5df24ef3c22a4aa8e9e9bc57b8511883f74b106844 WatchSource:0}: Error finding container 66b4253303c2a91c231c2b5df24ef3c22a4aa8e9e9bc57b8511883f74b106844: Status 404 returned error can't find the container with id 66b4253303c2a91c231c2b5df24ef3c22a4aa8e9e9bc57b8511883f74b106844
Aug 05 17:12:37.625120 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.625087    2574 apiserver.go:52] "Watching apiserver"
Aug 05 17:12:37.635152 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.635121    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Aug 05 17:12:37.636850 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.636824    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-hp8ck","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4","openshift-multus/multus-2pj44","openshift-multus/multus-additional-cni-plugins-phhjj","openshift-multus/network-metrics-daemon-zwhvz","openshift-network-operator/iptables-alerter-f9xdf","openshift-ovn-kubernetes/ovnkube-node-5bfsp","kube-system/global-pull-secret-syncer-8gjld","kube-system/konnectivity-agent-85lrg","kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal","openshift-cluster-node-tuning-operator/tuned-ndw7f","openshift-image-registry/node-ca-s5rqv","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal"]
Aug 05 17:12:37.639029 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.638996    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.640917 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.640891    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.642771 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.642747    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.643288 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.643270    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Aug 05 17:12:37.644583 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.644566    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.645619 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.645470    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Aug 05 17:12:37.645619 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.645479    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Aug 05 17:12:37.645619 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.645492    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Aug 05 17:12:37.645619 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.645571    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pzdzw\""
Aug 05 17:12:37.648510 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.648481    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:37.648640 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.648594    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:37.650435 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.650416    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.650616 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.650595    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Aug 05 17:12:37.651099 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.650897    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Aug 05 17:12:37.651099 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.650934    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tbcff\""
Aug 05 17:12:37.651975 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.651958    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Aug 05 17:12:37.652159 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.652141    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Aug 05 17:12:37.652231 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.652212    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Aug 05 17:12:37.652288 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.652142    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Aug 05 17:12:37.652346 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.652271    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rm5jf\""
Aug 05 17:12:37.652600 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.652583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:37.652696 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.652639    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:37.653259 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653237    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-daemon-config\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.653364 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653272    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-etc-openvswitch\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.653364 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-run-openvswitch\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.653364 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-tuning-conf-dir\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.653364 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.653573 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653381    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-var-lib-openvswitch\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.653573 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653408    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlblv\" (UniqueName: \"kubernetes.io/projected/f99723f8-ffe2-4e63-b860-1c1f670994d2-kube-api-access-xlblv\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.653573 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653437    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-var-lib-cni-bin\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.653573 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653478    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-run-multus-certs\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.653573 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653535    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-system-cni-dir\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.653573 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653551    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-cnibin\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.653573 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653573    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-log-socket\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653597    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-socket-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653615    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-socket-dir-parent\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653642    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-cni-binary-copy\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653706    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-run-systemd\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653730    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-run-ovn\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653756    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cab5a3d-0018-4593-8778-0b50c532bd1a-env-overrides\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653778    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-sys-fs\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653836    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-os-release\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653861    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cab5a3d-0018-4593-8778-0b50c532bd1a-ovn-node-metrics-cert\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.653899 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653888    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-registration-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.654467 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-run-netns\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.654467 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwgnr\" (UniqueName: \"kubernetes.io/projected/3bf16dd7-79db-4d4e-95f9-a863da309ada-kube-api-access-rwgnr\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.654467 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.653972    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-node-log\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.654467 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654063    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97tw8\" (UniqueName: \"kubernetes.io/projected/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-kube-api-access-97tw8\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.654467 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654167    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Aug 05 17:12:37.654467 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654254    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Aug 05 17:12:37.654878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654524    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Aug 05 17:12:37.654878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654531    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Aug 05 17:12:37.654878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654602    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:12:37.654878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654528    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pxqls\""
Aug 05 17:12:37.654878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654751    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-kubelet\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.654878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654793    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-systemd-units\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.654878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654836    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-slash\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.654878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654873    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Aug 05 17:12:37.654878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-run-netns\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654902    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-cni-dir\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654929    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3bf16dd7-79db-4d4e-95f9-a863da309ada-cni-binary-copy\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-os-release\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cab5a3d-0018-4593-8778-0b50c532bd1a-ovnkube-script-lib\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654985    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.654903    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655033    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-device-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-system-cni-dir\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655096    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-var-lib-kubelet\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655138    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-conf-dir\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655203    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-etc-kubernetes\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.655350 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655231    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.655582    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655268    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-cni-bin\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655663    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-cni-netd\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655688    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cab5a3d-0018-4593-8778-0b50c532bd1a-ovnkube-config\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655712    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-run-k8s-cni-cncf-io\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655734    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-hostroot\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655759    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-run-ovn-kubernetes\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655784    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jxdm\" (UniqueName: \"kubernetes.io/projected/5cab5a3d-0018-4593-8778-0b50c532bd1a-kube-api-access-2jxdm\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.655840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655831    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.656271 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655854    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-cnibin\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.656271 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.655898    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-var-lib-cni-multus\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.657224 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.657200    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:37.657552 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.657535    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5fh8h\""
Aug 05 17:12:37.657649 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.657587    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Aug 05 17:12:37.659450 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.659435    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.663239 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.662883    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-dz5h7\""
Aug 05 17:12:37.663239 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.663055    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:12:37.663239 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.663153    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Aug 05 17:12:37.663425 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.663076    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Aug 05 17:12:37.663425 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.663278    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Aug 05 17:12:37.663539 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.663471    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ktxc7\""
Aug 05 17:12:37.663539 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.663514    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.666558 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.666540    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Aug 05 17:12:37.666752 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.666735    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Aug 05 17:12:37.667131 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.667112    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tk49d\""
Aug 05 17:12:37.667219 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.667205    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Aug 05 17:12:37.672260 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.672238    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Aug 05 17:12:37.745763 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.745738    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Aug 05 17:12:37.757120 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.757262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-sys-fs\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.757262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757159    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-os-release\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.757262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757189    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-var-lib-kubelet\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.757262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757223    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/975a4dd9-7be5-40da-b2ce-ab8162b1d289-serviceca\") pod \"node-ca-s5rqv\" (UID: \"975a4dd9-7be5-40da-b2ce-ab8162b1d289\") " pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.757262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757250    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-node-log\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.757513 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757259    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.757513 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97tw8\" (UniqueName: \"kubernetes.io/projected/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-kube-api-access-97tw8\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.757513 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757318    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-systemd-units\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.757513 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757345    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-node-log\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.757513 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757349    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-sys-fs\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.757513 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757400    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-os-release\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.757513 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757404    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-systemd-units\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.757513 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757409    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-run-netns\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757579    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-run-netns\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757616    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3bf16dd7-79db-4d4e-95f9-a863da309ada-cni-binary-copy\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-conf-dir\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757683    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-etc-kubernetes\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757721    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdth9\" (UniqueName: \"kubernetes.io/projected/adeb3a83-c288-4d6e-a4ea-52d6c59b1a83-kube-api-access-hdth9\") pod \"iptables-alerter-f9xdf\" (UID: \"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83\") " pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-os-release\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cab5a3d-0018-4593-8778-0b50c532bd1a-ovnkube-script-lib\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-system-cni-dir\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757854    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adeb3a83-c288-4d6e-a4ea-52d6c59b1a83-host-slash\") pod \"iptables-alerter-f9xdf\" (UID: \"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83\") " pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/de68f860-081a-403c-8a3d-6299868bc425-kubelet-config\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757901    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-modprobe-d\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.757927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757926    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-tuned\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757948    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-cni-bin\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.757973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-cni-netd\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758004    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jxdm\" (UniqueName: \"kubernetes.io/projected/5cab5a3d-0018-4593-8778-0b50c532bd1a-kube-api-access-2jxdm\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758096    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-cnibin\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758130    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-var-lib-cni-multus\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758160    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-systemd\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-etc-openvswitch\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758220    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-run-openvswitch\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758243    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-var-lib-openvswitch\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758269    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlblv\" (UniqueName: \"kubernetes.io/projected/f99723f8-ffe2-4e63-b860-1c1f670994d2-kube-api-access-xlblv\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758294    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-var-lib-cni-bin\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f55cp\" (UniqueName: \"kubernetes.io/projected/975a4dd9-7be5-40da-b2ce-ab8162b1d289-kube-api-access-f55cp\") pod \"node-ca-s5rqv\" (UID: \"975a4dd9-7be5-40da-b2ce-ab8162b1d289\") " pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758322    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-os-release\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758357    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-cni-bin\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758398    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-cni-netd\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-cnibin\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.758463 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758432    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-etc-openvswitch\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758444    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-socket-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758468    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-cni-binary-copy\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758492    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-system-cni-dir\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758496    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-run-systemd\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758521    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-run-ovn\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758526    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-etc-kubernetes\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758545    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cab5a3d-0018-4593-8778-0b50c532bd1a-env-overrides\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758580    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e07bf2e-6270-40fc-9cb5-305b519cd00b-tmp\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758605    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbvx6\" (UniqueName: \"kubernetes.io/projected/1e07bf2e-6270-40fc-9cb5-305b519cd00b-kube-api-access-gbvx6\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758630    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cab5a3d-0018-4593-8778-0b50c532bd1a-ovn-node-metrics-cert\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758664    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-registration-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758691    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-run-netns\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.758733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwgnr\" (UniqueName: \"kubernetes.io/projected/3bf16dd7-79db-4d4e-95f9-a863da309ada-kube-api-access-rwgnr\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759087    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3bf16dd7-79db-4d4e-95f9-a863da309ada-cni-binary-copy\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-kubelet\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759159    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-cnibin\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759168    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-slash\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.759250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759195    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-cni-dir\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759223    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-var-lib-kubelet\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759251    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adeb3a83-c288-4d6e-a4ea-52d6c59b1a83-iptables-alerter-script\") pod \"iptables-alerter-f9xdf\" (UID: \"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83\") " pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759279    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-device-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759290    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-run-openvswitch\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759310    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b8083658-9899-4a1b-b99d-b97e1ad222b4-agent-certs\") pod \"konnectivity-agent-85lrg\" (UID: \"b8083658-9899-4a1b-b99d-b97e1ad222b4\") " pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759338    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759340    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-var-lib-openvswitch\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759378    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-var-lib-cni-bin\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759384    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-cnibin\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-conf-dir\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759400    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759410    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cab5a3d-0018-4593-8778-0b50c532bd1a-ovnkube-config\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759428    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-var-lib-cni-multus\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759435    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-run-netns\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759251    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5cab5a3d-0018-4593-8778-0b50c532bd1a-ovnkube-script-lib\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759483    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-var-lib-kubelet\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759499    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-socket-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.760095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759506    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-run-k8s-cni-cncf-io\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759531    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-kubelet\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759533    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-hostroot\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759568    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-hostroot\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759570    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759598    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/975a4dd9-7be5-40da-b2ce-ab8162b1d289-host\") pod \"node-ca-s5rqv\" (UID: \"975a4dd9-7be5-40da-b2ce-ab8162b1d289\") " pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759623    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759654    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-run-ovn-kubernetes\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759659    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-slash\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759661    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-registration-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759710    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-run-ovn\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759748    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-device-dir\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759711    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-run-ovn-kubernetes\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759775    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-run-systemd\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759795    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759824    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759852    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-daemon-config\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.760856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759866    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-cni-dir\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759900    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/de68f860-081a-403c-8a3d-6299868bc425-dbus\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759925    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759943    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f99723f8-ffe2-4e63-b860-1c1f670994d2-etc-selinux\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b8083658-9899-4a1b-b99d-b97e1ad222b4-konnectivity-ca\") pod \"konnectivity-agent-85lrg\" (UID: \"b8083658-9899-4a1b-b99d-b97e1ad222b4\") " pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.759984    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-run-k8s-cni-cncf-io\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760027    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5cab5a3d-0018-4593-8778-0b50c532bd1a-ovnkube-config\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-tuning-conf-dir\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760058    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-cni-binary-copy\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760073    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760095    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760109    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-run-multus-certs\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760149    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5cab5a3d-0018-4593-8778-0b50c532bd1a-env-overrides\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760159    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-host-run-multus-certs\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760189    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-sysconfig\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760197    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760231    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-sysctl-conf\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.761560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760267    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-run\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760271    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-tuning-conf-dir\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760293    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-sys\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760317    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-lib-modules\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760352    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-system-cni-dir\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760391    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-system-cni-dir\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760393    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-log-socket\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760433    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-socket-dir-parent\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760436    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5cab5a3d-0018-4593-8778-0b50c532bd1a-log-socket\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760469    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh62n\" (UniqueName: \"kubernetes.io/projected/e3507e66-54c4-4511-ad30-6ec595a04bd4-kube-api-access-mh62n\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760476    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-daemon-config\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760485    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3bf16dd7-79db-4d4e-95f9-a863da309ada-multus-socket-dir-parent\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760520    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-kubernetes\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760548    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-sysctl-d\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760572    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-host\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.762401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.760593    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.763375 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.763318    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5cab5a3d-0018-4593-8778-0b50c532bd1a-ovn-node-metrics-cert\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.768446 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.767991    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97tw8\" (UniqueName: \"kubernetes.io/projected/50db4820-71dc-48e9-ba6c-9ddd8f32eb01-kube-api-access-97tw8\") pod \"multus-additional-cni-plugins-phhjj\" (UID: \"50db4820-71dc-48e9-ba6c-9ddd8f32eb01\") " pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.768559 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.768459    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlblv\" (UniqueName: \"kubernetes.io/projected/f99723f8-ffe2-4e63-b860-1c1f670994d2-kube-api-access-xlblv\") pod \"aws-ebs-csi-driver-node-2rhm4\" (UID: \"f99723f8-ffe2-4e63-b860-1c1f670994d2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.772009 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.771964    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jxdm\" (UniqueName: \"kubernetes.io/projected/5cab5a3d-0018-4593-8778-0b50c532bd1a-kube-api-access-2jxdm\") pod \"ovnkube-node-5bfsp\" (UID: \"5cab5a3d-0018-4593-8778-0b50c532bd1a\") " pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.775790 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.775746    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwgnr\" (UniqueName: \"kubernetes.io/projected/3bf16dd7-79db-4d4e-95f9-a863da309ada-kube-api-access-rwgnr\") pod \"multus-2pj44\" (UID: \"3bf16dd7-79db-4d4e-95f9-a863da309ada\") " pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.795116 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.795095    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-n22jk"
Aug 05 17:12:37.804953 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.804070    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-n22jk"
Aug 05 17:12:37.824662 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.824640    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:37.855040 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.854937    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal" event={"ID":"0ea1cdff287318a9cf0bdf169156a32c","Type":"ContainerStarted","Data":"66b4253303c2a91c231c2b5df24ef3c22a4aa8e9e9bc57b8511883f74b106844"}
Aug 05 17:12:37.857062 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.857036    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal" event={"ID":"80068e53ca81b677a6b8796d1e4de03a","Type":"ContainerStarted","Data":"28187a12c2267b5a380f675281f6c67e6e8a2a4455fbc382089b5a335d3db600"}
Aug 05 17:12:37.861592 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdth9\" (UniqueName: \"kubernetes.io/projected/adeb3a83-c288-4d6e-a4ea-52d6c59b1a83-kube-api-access-hdth9\") pod \"iptables-alerter-f9xdf\" (UID: \"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83\") " pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.861695 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861608    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adeb3a83-c288-4d6e-a4ea-52d6c59b1a83-host-slash\") pod \"iptables-alerter-f9xdf\" (UID: \"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83\") " pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.861695 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/de68f860-081a-403c-8a3d-6299868bc425-kubelet-config\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:37.861695 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861660    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-modprobe-d\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.861695 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861683    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-tuned\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861710    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-systemd\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861736    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f55cp\" (UniqueName: \"kubernetes.io/projected/975a4dd9-7be5-40da-b2ce-ab8162b1d289-kube-api-access-f55cp\") pod \"node-ca-s5rqv\" (UID: \"975a4dd9-7be5-40da-b2ce-ab8162b1d289\") " pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861762    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e07bf2e-6270-40fc-9cb5-305b519cd00b-tmp\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861784    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbvx6\" (UniqueName: \"kubernetes.io/projected/1e07bf2e-6270-40fc-9cb5-305b519cd00b-kube-api-access-gbvx6\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adeb3a83-c288-4d6e-a4ea-52d6c59b1a83-iptables-alerter-script\") pod \"iptables-alerter-f9xdf\" (UID: \"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83\") " pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861837    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b8083658-9899-4a1b-b99d-b97e1ad222b4-agent-certs\") pod \"konnectivity-agent-85lrg\" (UID: \"b8083658-9899-4a1b-b99d-b97e1ad222b4\") " pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861864    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861885    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/975a4dd9-7be5-40da-b2ce-ab8162b1d289-host\") pod \"node-ca-s5rqv\" (UID: \"975a4dd9-7be5-40da-b2ce-ab8162b1d289\") " pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861908    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-systemd\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.861928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861910    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861968    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/de68f860-081a-403c-8a3d-6299868bc425-dbus\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.861994    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.862005    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b8083658-9899-4a1b-b99d-b97e1ad222b4-konnectivity-ca\") pod \"konnectivity-agent-85lrg\" (UID: \"b8083658-9899-4a1b-b99d-b97e1ad222b4\") " pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862069    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-sysconfig\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.862090    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs podName:e3507e66-54c4-4511-ad30-6ec595a04bd4 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:38.362070194 +0000 UTC m=+3.061325523 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs") pod "network-metrics-daemon-zwhvz" (UID: "e3507e66-54c4-4511-ad30-6ec595a04bd4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862117    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-sysconfig\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862116    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-sysctl-conf\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862153    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-run\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862178    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-sys\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862200    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-lib-modules\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862230    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh62n\" (UniqueName: \"kubernetes.io/projected/e3507e66-54c4-4511-ad30-6ec595a04bd4-kube-api-access-mh62n\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.862246    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862254    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-kubernetes\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862277    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-sysctl-d\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.862302    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret podName:de68f860-081a-403c-8a3d-6299868bc425 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:38.362285947 +0000 UTC m=+3.061541271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret") pod "global-pull-secret-syncer-8gjld" (UID: "de68f860-081a-403c-8a3d-6299868bc425") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:37.862532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-host\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862363    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-var-lib-kubelet\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/975a4dd9-7be5-40da-b2ce-ab8162b1d289-serviceca\") pod \"node-ca-s5rqv\" (UID: \"975a4dd9-7be5-40da-b2ce-ab8162b1d289\") " pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862414    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-sysctl-d\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862483    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-host\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862695    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-var-lib-kubelet\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862744    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/de68f860-081a-403c-8a3d-6299868bc425-kubelet-config\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862797    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/adeb3a83-c288-4d6e-a4ea-52d6c59b1a83-host-slash\") pod \"iptables-alerter-f9xdf\" (UID: \"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83\") " pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-modprobe-d\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862908    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-sysctl-conf\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862917    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-lib-modules\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.863117    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-run\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.863173    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-sys\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.862178    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/de68f860-081a-403c-8a3d-6299868bc425-dbus\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.863212    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/975a4dd9-7be5-40da-b2ce-ab8162b1d289-host\") pod \"node-ca-s5rqv\" (UID: \"975a4dd9-7be5-40da-b2ce-ab8162b1d289\") " pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.863362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.863317    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-kubernetes\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.864120 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.863672    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/adeb3a83-c288-4d6e-a4ea-52d6c59b1a83-iptables-alerter-script\") pod \"iptables-alerter-f9xdf\" (UID: \"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83\") " pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.864120 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.863975    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/975a4dd9-7be5-40da-b2ce-ab8162b1d289-serviceca\") pod \"node-ca-s5rqv\" (UID: \"975a4dd9-7be5-40da-b2ce-ab8162b1d289\") " pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.865238 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.865214    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b8083658-9899-4a1b-b99d-b97e1ad222b4-konnectivity-ca\") pod \"konnectivity-agent-85lrg\" (UID: \"b8083658-9899-4a1b-b99d-b97e1ad222b4\") " pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:37.865717 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.865692    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1e07bf2e-6270-40fc-9cb5-305b519cd00b-etc-tuned\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.866804 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.866764    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b8083658-9899-4a1b-b99d-b97e1ad222b4-agent-certs\") pod \"konnectivity-agent-85lrg\" (UID: \"b8083658-9899-4a1b-b99d-b97e1ad222b4\") " pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:37.867136 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.867116    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1e07bf2e-6270-40fc-9cb5-305b519cd00b-tmp\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.871408 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.870958    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:37.871408 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.870979    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:37.871408 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.870993    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h54dv for pod openshift-network-diagnostics/network-check-target-hp8ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:37.871408 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:37.871063    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv podName:25c6e85a-9ed2-446f-9b1f-fa204867ac7a nodeName:}" failed. No retries permitted until 2026-08-05 17:12:38.371045445 +0000 UTC m=+3.070300768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-h54dv" (UniqueName: "kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv") pod "network-check-target-hp8ck" (UID: "25c6e85a-9ed2-446f-9b1f-fa204867ac7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:37.872615 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.872592    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdth9\" (UniqueName: \"kubernetes.io/projected/adeb3a83-c288-4d6e-a4ea-52d6c59b1a83-kube-api-access-hdth9\") pod \"iptables-alerter-f9xdf\" (UID: \"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83\") " pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.874366 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.874329    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f55cp\" (UniqueName: \"kubernetes.io/projected/975a4dd9-7be5-40da-b2ce-ab8162b1d289-kube-api-access-f55cp\") pod \"node-ca-s5rqv\" (UID: \"975a4dd9-7be5-40da-b2ce-ab8162b1d289\") " pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:37.874892 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.874868    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh62n\" (UniqueName: \"kubernetes.io/projected/e3507e66-54c4-4511-ad30-6ec595a04bd4-kube-api-access-mh62n\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:37.875310 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.875262    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbvx6\" (UniqueName: \"kubernetes.io/projected/1e07bf2e-6270-40fc-9cb5-305b519cd00b-kube-api-access-gbvx6\") pod \"tuned-ndw7f\" (UID: \"1e07bf2e-6270-40fc-9cb5-305b519cd00b\") " pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.950633 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.950608    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:12:37.958925 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.958895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2pj44"
Aug 05 17:12:37.968765 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.968746    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4"
Aug 05 17:12:37.973444 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.973423    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-phhjj"
Aug 05 17:12:37.980674 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.980647    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f9xdf"
Aug 05 17:12:37.987700 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.987683    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:37.994329 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.994189    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ndw7f"
Aug 05 17:12:37.998811 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:37.998791    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s5rqv"
Aug 05 17:12:38.155908 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:38.155875    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadeb3a83_c288_4d6e_a4ea_52d6c59b1a83.slice/crio-66b63e624de68b370caa5b6024f5ee41ee9f599fd7dc28771b6be2f7d28ed6d6 WatchSource:0}: Error finding container 66b63e624de68b370caa5b6024f5ee41ee9f599fd7dc28771b6be2f7d28ed6d6: Status 404 returned error can't find the container with id 66b63e624de68b370caa5b6024f5ee41ee9f599fd7dc28771b6be2f7d28ed6d6
Aug 05 17:12:38.157133 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:38.157110    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50db4820_71dc_48e9_ba6c_9ddd8f32eb01.slice/crio-46bab6c6d6e79b55db080133b97135a481a86824bedfc3250842585e4cccf48a WatchSource:0}: Error finding container 46bab6c6d6e79b55db080133b97135a481a86824bedfc3250842585e4cccf48a: Status 404 returned error can't find the container with id 46bab6c6d6e79b55db080133b97135a481a86824bedfc3250842585e4cccf48a
Aug 05 17:12:38.160844 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:38.160815    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf16dd7_79db_4d4e_95f9_a863da309ada.slice/crio-307d0fb89cdad9140accde1266f9a78ab2854e6c72c39c147fc334d8e5b00683 WatchSource:0}: Error finding container 307d0fb89cdad9140accde1266f9a78ab2854e6c72c39c147fc334d8e5b00683: Status 404 returned error can't find the container with id 307d0fb89cdad9140accde1266f9a78ab2854e6c72c39c147fc334d8e5b00683
Aug 05 17:12:38.164093 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:38.164066    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8083658_9899_4a1b_b99d_b97e1ad222b4.slice/crio-ee77c6a378ca60869fca5b15ace376b2b33e2dd70e63045908512873755df159 WatchSource:0}: Error finding container ee77c6a378ca60869fca5b15ace376b2b33e2dd70e63045908512873755df159: Status 404 returned error can't find the container with id ee77c6a378ca60869fca5b15ace376b2b33e2dd70e63045908512873755df159
Aug 05 17:12:38.183544 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:38.183520    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cab5a3d_0018_4593_8778_0b50c532bd1a.slice/crio-4c70b4e5a36bb7c051de55f51a9ccf5ee37c478194c2c5b7e5b4a0a1dee268b3 WatchSource:0}: Error finding container 4c70b4e5a36bb7c051de55f51a9ccf5ee37c478194c2c5b7e5b4a0a1dee268b3: Status 404 returned error can't find the container with id 4c70b4e5a36bb7c051de55f51a9ccf5ee37c478194c2c5b7e5b4a0a1dee268b3
Aug 05 17:12:38.184364 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:38.184293    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99723f8_ffe2_4e63_b860_1c1f670994d2.slice/crio-efdb5907ab35712023eeed142908b79c3877ac3fceea3d1d385abb46c2a7c809 WatchSource:0}: Error finding container efdb5907ab35712023eeed142908b79c3877ac3fceea3d1d385abb46c2a7c809: Status 404 returned error can't find the container with id efdb5907ab35712023eeed142908b79c3877ac3fceea3d1d385abb46c2a7c809
Aug 05 17:12:38.185050 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:38.184923    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e07bf2e_6270_40fc_9cb5_305b519cd00b.slice/crio-05dd1bbd46f9c1a23ef5c1088329526a1e3fe17633d523abad05c99282643715 WatchSource:0}: Error finding container 05dd1bbd46f9c1a23ef5c1088329526a1e3fe17633d523abad05c99282643715: Status 404 returned error can't find the container with id 05dd1bbd46f9c1a23ef5c1088329526a1e3fe17633d523abad05c99282643715
Aug 05 17:12:38.185779 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:38.185760    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975a4dd9_7be5_40da_b2ce_ab8162b1d289.slice/crio-684ebf3c1473c4f2b2f45ff1bf19590f87244c7b71d4a445751d79f4ba71101f WatchSource:0}: Error finding container 684ebf3c1473c4f2b2f45ff1bf19590f87244c7b71d4a445751d79f4ba71101f: Status 404 returned error can't find the container with id 684ebf3c1473c4f2b2f45ff1bf19590f87244c7b71d4a445751d79f4ba71101f
Aug 05 17:12:38.366567 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.366535    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:38.366567 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.366569    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:38.366746 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.366663    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:38.366746 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.366689    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:38.366746 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.366724    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret podName:de68f860-081a-403c-8a3d-6299868bc425 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:39.366710839 +0000 UTC m=+4.065966183 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret") pod "global-pull-secret-syncer-8gjld" (UID: "de68f860-081a-403c-8a3d-6299868bc425") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:38.366907 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.366749    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs podName:e3507e66-54c4-4511-ad30-6ec595a04bd4 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:39.366730739 +0000 UTC m=+4.065986064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs") pod "network-metrics-daemon-zwhvz" (UID: "e3507e66-54c4-4511-ad30-6ec595a04bd4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:38.467528 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.467451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:38.467678 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.467632    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:38.467678 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.467653    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:38.467678 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.467664    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h54dv for pod openshift-network-diagnostics/network-check-target-hp8ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:38.467849 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.467719    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv podName:25c6e85a-9ed2-446f-9b1f-fa204867ac7a nodeName:}" failed. No retries permitted until 2026-08-05 17:12:39.467701444 +0000 UTC m=+4.166956785 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-h54dv" (UniqueName: "kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv") pod "network-check-target-hp8ck" (UID: "25c6e85a-9ed2-446f-9b1f-fa204867ac7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:38.809310 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.809226    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:07:37 +0000 UTC" deadline="2028-05-10 21:07:11.932857084 +0000 UTC"
Aug 05 17:12:38.809310 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.809272    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15459h54m33.123589693s"
Aug 05 17:12:38.849065 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.849043    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:38.849165 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.849148    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:38.849226 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.849212    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:38.849347 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:38.849318    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:38.859740 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.859722    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" event={"ID":"5cab5a3d-0018-4593-8778-0b50c532bd1a","Type":"ContainerStarted","Data":"4c70b4e5a36bb7c051de55f51a9ccf5ee37c478194c2c5b7e5b4a0a1dee268b3"}
Aug 05 17:12:38.860907 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.860888    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4" event={"ID":"f99723f8-ffe2-4e63-b860-1c1f670994d2","Type":"ContainerStarted","Data":"efdb5907ab35712023eeed142908b79c3877ac3fceea3d1d385abb46c2a7c809"}
Aug 05 17:12:38.864629 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.862513    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s5rqv" event={"ID":"975a4dd9-7be5-40da-b2ce-ab8162b1d289","Type":"ContainerStarted","Data":"684ebf3c1473c4f2b2f45ff1bf19590f87244c7b71d4a445751d79f4ba71101f"}
Aug 05 17:12:38.866066 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.866041    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-85lrg" event={"ID":"b8083658-9899-4a1b-b99d-b97e1ad222b4","Type":"ContainerStarted","Data":"ee77c6a378ca60869fca5b15ace376b2b33e2dd70e63045908512873755df159"}
Aug 05 17:12:38.867104 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.867081    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2pj44" event={"ID":"3bf16dd7-79db-4d4e-95f9-a863da309ada","Type":"ContainerStarted","Data":"307d0fb89cdad9140accde1266f9a78ab2854e6c72c39c147fc334d8e5b00683"}
Aug 05 17:12:38.868235 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.868209    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-phhjj" event={"ID":"50db4820-71dc-48e9-ba6c-9ddd8f32eb01","Type":"ContainerStarted","Data":"46bab6c6d6e79b55db080133b97135a481a86824bedfc3250842585e4cccf48a"}
Aug 05 17:12:38.869412 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.869345    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ndw7f" event={"ID":"1e07bf2e-6270-40fc-9cb5-305b519cd00b","Type":"ContainerStarted","Data":"05dd1bbd46f9c1a23ef5c1088329526a1e3fe17633d523abad05c99282643715"}
Aug 05 17:12:38.870810 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:38.870790    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f9xdf" event={"ID":"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83","Type":"ContainerStarted","Data":"66b63e624de68b370caa5b6024f5ee41ee9f599fd7dc28771b6be2f7d28ed6d6"}
Aug 05 17:12:39.374958 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.374924    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:39.375120 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.374977    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:39.375184 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:39.375138    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:39.375247 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:39.375138    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:39.375247 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:39.375220    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret podName:de68f860-081a-403c-8a3d-6299868bc425 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:41.375202731 +0000 UTC m=+6.074458052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret") pod "global-pull-secret-syncer-8gjld" (UID: "de68f860-081a-403c-8a3d-6299868bc425") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:39.375363 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:39.375251    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs podName:e3507e66-54c4-4511-ad30-6ec595a04bd4 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:41.37523946 +0000 UTC m=+6.074494789 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs") pod "network-metrics-daemon-zwhvz" (UID: "e3507e66-54c4-4511-ad30-6ec595a04bd4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:39.476762 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.475983    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:39.476762 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:39.476217    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:39.476762 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:39.476241    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:39.476762 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:39.476255    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h54dv for pod openshift-network-diagnostics/network-check-target-hp8ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:39.476762 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:39.476312    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv podName:25c6e85a-9ed2-446f-9b1f-fa204867ac7a nodeName:}" failed. No retries permitted until 2026-08-05 17:12:41.47629429 +0000 UTC m=+6.175549628 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-h54dv" (UniqueName: "kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv") pod "network-check-target-hp8ck" (UID: "25c6e85a-9ed2-446f-9b1f-fa204867ac7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:39.525915 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.525887    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:39.718724 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.718687    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Aug 05 17:12:39.809523 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.809429    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-08-04 17:07:37 +0000 UTC" deadline="2028-01-08 16:11:14.321552665 +0000 UTC"
Aug 05 17:12:39.809523 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.809483    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12502h58m34.512074208s"
Aug 05 17:12:39.851041 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.850999    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:39.851192 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:39.851142    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:39.885282 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.885248    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal" event={"ID":"0ea1cdff287318a9cf0bdf169156a32c","Type":"ContainerStarted","Data":"816f36ae77e5dbdc84f8b4366c4474a53a90262cf040a3b74246c32c7ad02301"}
Aug 05 17:12:39.900045 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.897079    2574 generic.go:356] "Generic (PLEG): container finished" podID="80068e53ca81b677a6b8796d1e4de03a" containerID="d5584d9210da449624637a2c88f4bf01698a4a3d015d54274b48f1d3e0be8dff" exitCode=0
Aug 05 17:12:39.900045 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.897139    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal" event={"ID":"80068e53ca81b677a6b8796d1e4de03a","Type":"ContainerDied","Data":"d5584d9210da449624637a2c88f4bf01698a4a3d015d54274b48f1d3e0be8dff"}
Aug 05 17:12:39.927718 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:39.926551    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-246.ec2.internal" podStartSLOduration=3.926531937 podStartE2EDuration="3.926531937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:39.903921537 +0000 UTC m=+4.603176880" watchObservedRunningTime="2026-08-05 17:12:39.926531937 +0000 UTC m=+4.625787279"
Aug 05 17:12:40.849870 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:40.849836    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:40.850546 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:40.850009    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:40.850546 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:40.850434    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:40.850546 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:40.850528    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:40.917967 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:40.917930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal" event={"ID":"80068e53ca81b677a6b8796d1e4de03a","Type":"ContainerStarted","Data":"499075c66fc21860f1e3be6e4620c04e69d11de1b21e41e2737605949cf77312"}
Aug 05 17:12:41.394553 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:41.393819    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:41.394553 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:41.393869    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:41.394553 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:41.394031    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:41.394553 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:41.394086    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret podName:de68f860-081a-403c-8a3d-6299868bc425 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:45.394069452 +0000 UTC m=+10.093324774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret") pod "global-pull-secret-syncer-8gjld" (UID: "de68f860-081a-403c-8a3d-6299868bc425") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:41.394553 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:41.394470    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:41.394553 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:41.394521    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs podName:e3507e66-54c4-4511-ad30-6ec595a04bd4 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:45.394505354 +0000 UTC m=+10.093760674 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs") pod "network-metrics-daemon-zwhvz" (UID: "e3507e66-54c4-4511-ad30-6ec595a04bd4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:41.494826 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:41.494791    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:41.494984 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:41.494962    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:41.494984 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:41.494979    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:41.495102 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:41.494990    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h54dv for pod openshift-network-diagnostics/network-check-target-hp8ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:41.495152 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:41.495139    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv podName:25c6e85a-9ed2-446f-9b1f-fa204867ac7a nodeName:}" failed. No retries permitted until 2026-08-05 17:12:45.49508642 +0000 UTC m=+10.194341759 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-h54dv" (UniqueName: "kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv") pod "network-check-target-hp8ck" (UID: "25c6e85a-9ed2-446f-9b1f-fa204867ac7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:41.851426 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:41.851351    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:41.851863 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:41.851498    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:42.849515 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:42.849486    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:42.849703 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:42.849602    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:42.849703 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:42.849675    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:42.849933 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:42.849741    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:43.850537 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:43.849895    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:43.850537 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:43.850057    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:44.849990 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:44.849333    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:44.849990 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:44.849464    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:44.849990 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:44.849849    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:44.849990 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:44.849946    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:45.428895 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:45.428863    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:45.429375 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:45.428916    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:45.429375 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:45.429002    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:45.429375 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:45.429087    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs podName:e3507e66-54c4-4511-ad30-6ec595a04bd4 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:53.429065735 +0000 UTC m=+18.128321277 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs") pod "network-metrics-daemon-zwhvz" (UID: "e3507e66-54c4-4511-ad30-6ec595a04bd4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:45.429375 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:45.429094    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:45.429375 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:45.429162    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret podName:de68f860-081a-403c-8a3d-6299868bc425 nodeName:}" failed. No retries permitted until 2026-08-05 17:12:53.429142982 +0000 UTC m=+18.128398302 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret") pod "global-pull-secret-syncer-8gjld" (UID: "de68f860-081a-403c-8a3d-6299868bc425") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:45.530435 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:45.529973    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:45.530435 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:45.530181    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:45.530435 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:45.530201    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:45.530435 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:45.530213    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h54dv for pod openshift-network-diagnostics/network-check-target-hp8ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:45.530435 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:45.530278    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv podName:25c6e85a-9ed2-446f-9b1f-fa204867ac7a nodeName:}" failed. No retries permitted until 2026-08-05 17:12:53.530256979 +0000 UTC m=+18.229512321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-h54dv" (UniqueName: "kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv") pod "network-check-target-hp8ck" (UID: "25c6e85a-9ed2-446f-9b1f-fa204867ac7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:45.850780 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:45.850228    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:45.850780 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:45.850365    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:46.850051 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:46.849581    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:46.850051 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:46.849706    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:46.850051 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:46.849982    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:46.850641 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:46.850108    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:47.849537 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:47.849504    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:47.849708 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:47.849641    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:48.849716 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:48.849687    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:48.850099 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:48.849687    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:48.850099 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:48.849774    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:48.850099 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:48.849842    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:49.853348 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:49.853319    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:49.853754 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:49.853432    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:50.849262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:50.849228    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:50.849262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:50.849249    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:50.849488 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:50.849360    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:50.849488 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:50.849466    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:51.849706 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:51.849680    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:51.850169 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:51.849817    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:52.849545 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:52.849460    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:52.849545 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:52.849483    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:52.849784 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:52.849600    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:52.849784 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:52.849740    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:53.486933 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:53.486895    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:53.487124 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:53.486949    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:53.487124 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:53.487066    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:53.487124 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:53.487102    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:53.487124 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:53.487126    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs podName:e3507e66-54c4-4511-ad30-6ec595a04bd4 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:09.487110574 +0000 UTC m=+34.186365894 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs") pod "network-metrics-daemon-zwhvz" (UID: "e3507e66-54c4-4511-ad30-6ec595a04bd4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:12:53.487298 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:53.487163    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret podName:de68f860-081a-403c-8a3d-6299868bc425 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:09.487144939 +0000 UTC m=+34.186400261 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret") pod "global-pull-secret-syncer-8gjld" (UID: "de68f860-081a-403c-8a3d-6299868bc425") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:12:53.587611 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:53.587578    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:53.587802 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:53.587721    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:12:53.587802 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:53.587736    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:12:53.587802 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:53.587746    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h54dv for pod openshift-network-diagnostics/network-check-target-hp8ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:53.587802 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:53.587800    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv podName:25c6e85a-9ed2-446f-9b1f-fa204867ac7a nodeName:}" failed. No retries permitted until 2026-08-05 17:13:09.58778418 +0000 UTC m=+34.287039503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-h54dv" (UniqueName: "kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv") pod "network-check-target-hp8ck" (UID: "25c6e85a-9ed2-446f-9b1f-fa204867ac7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:12:53.849353 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:53.849282    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:53.849504 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:53.849429    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:54.849265 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:54.849231    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:54.849660 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:54.849242    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:54.849660 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:54.849333    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:54.849660 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:54.849445    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:55.800387 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.800339    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-246.ec2.internal" podStartSLOduration=19.800323545 podStartE2EDuration="19.800323545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:40.942330999 +0000 UTC m=+5.641586342" watchObservedRunningTime="2026-08-05 17:12:55.800323545 +0000 UTC m=+20.499578886"
Aug 05 17:12:55.800736 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.800720    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-llnmn"]
Aug 05 17:12:55.807901 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.807882    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:55.812201 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.812173    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Aug 05 17:12:55.812312 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.812261    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cfcdk\""
Aug 05 17:12:55.812312 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.812302    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Aug 05 17:12:55.849832 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.849807    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:55.850200 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:55.849908    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:55.905269 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.905235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4-tmp-dir\") pod \"node-resolver-llnmn\" (UID: \"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4\") " pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:55.905389 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.905287    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4-hosts-file\") pod \"node-resolver-llnmn\" (UID: \"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4\") " pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:55.905445 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:55.905404    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwm9n\" (UniqueName: \"kubernetes.io/projected/cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4-kube-api-access-zwm9n\") pod \"node-resolver-llnmn\" (UID: \"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4\") " pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:56.006061 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.006026    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4-hosts-file\") pod \"node-resolver-llnmn\" (UID: \"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4\") " pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:56.006275 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.006114    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4-hosts-file\") pod \"node-resolver-llnmn\" (UID: \"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4\") " pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:56.006275 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.006124    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwm9n\" (UniqueName: \"kubernetes.io/projected/cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4-kube-api-access-zwm9n\") pod \"node-resolver-llnmn\" (UID: \"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4\") " pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:56.006275 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.006191    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4-tmp-dir\") pod \"node-resolver-llnmn\" (UID: \"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4\") " pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:56.006517 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.006499    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4-tmp-dir\") pod \"node-resolver-llnmn\" (UID: \"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4\") " pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:56.016386 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.016361    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwm9n\" (UniqueName: \"kubernetes.io/projected/cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4-kube-api-access-zwm9n\") pod \"node-resolver-llnmn\" (UID: \"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4\") " pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:56.117539 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.117474    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-llnmn"
Aug 05 17:12:56.849378 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.849232    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:56.849378 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:56.849338    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:56.849585 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.849391    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:56.849585 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:56.849461    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:56.909700 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:12:56.909670    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0aaa1f_a8d6_41c5_8f22_c5bde34131a4.slice/crio-df97aa7065241364cc07476dabe0e0d19f87d6de471543ba4dcaa63762757b75 WatchSource:0}: Error finding container df97aa7065241364cc07476dabe0e0d19f87d6de471543ba4dcaa63762757b75: Status 404 returned error can't find the container with id df97aa7065241364cc07476dabe0e0d19f87d6de471543ba4dcaa63762757b75
Aug 05 17:12:56.948929 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:56.948903    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-llnmn" event={"ID":"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4","Type":"ContainerStarted","Data":"df97aa7065241364cc07476dabe0e0d19f87d6de471543ba4dcaa63762757b75"}
Aug 05 17:12:57.849532 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.849351    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:57.849664 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:57.849620    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:57.953536 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.953470    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" event={"ID":"5cab5a3d-0018-4593-8778-0b50c532bd1a","Type":"ContainerStarted","Data":"19e4df14e30ea01e1349b2cb586349c2cb9bb9ebb4f746da6ff1834465aa6d04"}
Aug 05 17:12:57.953536 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.953530    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" event={"ID":"5cab5a3d-0018-4593-8778-0b50c532bd1a","Type":"ContainerStarted","Data":"81a8236f3d35c1d61f17fe4976db00bbe823d3010dfbe3c095178959f2610b38"}
Aug 05 17:12:57.954290 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.953569    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" event={"ID":"5cab5a3d-0018-4593-8778-0b50c532bd1a","Type":"ContainerStarted","Data":"aa0e70ac68e5ebc4530c29c88913c8d3a905768aba510a7d0425b4c2b0f353b9"}
Aug 05 17:12:57.954290 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.953603    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" event={"ID":"5cab5a3d-0018-4593-8778-0b50c532bd1a","Type":"ContainerStarted","Data":"e9a6abc898f72694193e30a73d64af113932a2fbf94671e14bb8ffd81a63635b"}
Aug 05 17:12:57.954290 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.953638    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" event={"ID":"5cab5a3d-0018-4593-8778-0b50c532bd1a","Type":"ContainerStarted","Data":"af5269c8c4d97a66e1bfaa5bf7a2d95d7ce898d1a6395a388f8c874faf2b58ab"}
Aug 05 17:12:57.954290 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.953665    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" event={"ID":"5cab5a3d-0018-4593-8778-0b50c532bd1a","Type":"ContainerStarted","Data":"2fd4c32d355c5eb1ab06d30f9d1a4c1860867fd1b42b3914a8537b5e3b325642"}
Aug 05 17:12:57.954791 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.954770    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4" event={"ID":"f99723f8-ffe2-4e63-b860-1c1f670994d2","Type":"ContainerStarted","Data":"bdf575a9722cc9abd7fb812854ae1b5558be8300c6424740aa8cbf153d9f17a9"}
Aug 05 17:12:57.955941 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.955924    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s5rqv" event={"ID":"975a4dd9-7be5-40da-b2ce-ab8162b1d289","Type":"ContainerStarted","Data":"71ffe2d5393a5dbdf4ef38231d4cd04f8c94a57cddac499fc36f62f9694f445c"}
Aug 05 17:12:57.957283 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.957251    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-85lrg" event={"ID":"b8083658-9899-4a1b-b99d-b97e1ad222b4","Type":"ContainerStarted","Data":"aef2ce9392c6b88902dc46ba7a04fed6b2fdbd61f8f14354a1a51a46060ebfef"}
Aug 05 17:12:57.958573 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.958552    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2pj44" event={"ID":"3bf16dd7-79db-4d4e-95f9-a863da309ada","Type":"ContainerStarted","Data":"9240c121c0d3ccd5cd5bab49279caa4199d4c5d249a3a9428869ae6279bc8407"}
Aug 05 17:12:57.959983 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.959960    2574 generic.go:356] "Generic (PLEG): container finished" podID="50db4820-71dc-48e9-ba6c-9ddd8f32eb01" containerID="9683b15c8a7dc5cdc9fb1e49fb1849cbff5ca0e0cf17bc4dafddd6e892232ae1" exitCode=0
Aug 05 17:12:57.960091 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.960063    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-phhjj" event={"ID":"50db4820-71dc-48e9-ba6c-9ddd8f32eb01","Type":"ContainerDied","Data":"9683b15c8a7dc5cdc9fb1e49fb1849cbff5ca0e0cf17bc4dafddd6e892232ae1"}
Aug 05 17:12:57.962250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.961713    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ndw7f" event={"ID":"1e07bf2e-6270-40fc-9cb5-305b519cd00b","Type":"ContainerStarted","Data":"80f26062a2526e16a31ff1df7b26b7383a922cbd60eedee3c3ad8293bc931dd9"}
Aug 05 17:12:57.966096 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.966064    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-llnmn" event={"ID":"cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4","Type":"ContainerStarted","Data":"8aef9046b85953e479083b19f548a38074fc28308aa602b6088c37519c5f6cdf"}
Aug 05 17:12:57.988669 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.988627    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-llnmn" podStartSLOduration=2.9886139480000002 podStartE2EDuration="2.988613948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:57.988149885 +0000 UTC m=+22.687405228" watchObservedRunningTime="2026-08-05 17:12:57.988613948 +0000 UTC m=+22.687869289"
Aug 05 17:12:57.988897 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:57.988870    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s5rqv" podStartSLOduration=4.273201586 podStartE2EDuration="22.988862391s" totalImagesPullingTime="18.715660805s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:57.972138884 +0000 UTC m=+22.671394222" watchObservedRunningTime="2026-08-05 17:12:57.988862391 +0000 UTC m=+22.688117732"
Aug 05 17:12:58.007044 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:58.006987    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2pj44" podStartSLOduration=4.255976051 podStartE2EDuration="23.006972053s" totalImagesPullingTime="18.750996002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:58.006961257 +0000 UTC m=+22.706216599" watchObservedRunningTime="2026-08-05 17:12:58.006972053 +0000 UTC m=+22.706227398"
Aug 05 17:12:58.050632 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:58.050578    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-85lrg" podStartSLOduration=8.656130418 podStartE2EDuration="23.050561431s" totalImagesPullingTime="14.394431013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:58.049708539 +0000 UTC m=+22.748963883" watchObservedRunningTime="2026-08-05 17:12:58.050561431 +0000 UTC m=+22.749816804"
Aug 05 17:12:58.069327 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:58.069289    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ndw7f" podStartSLOduration=4.3514574679999996 podStartE2EDuration="23.069277911s" totalImagesPullingTime="18.717820443s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:58.069227635 +0000 UTC m=+22.768482978" watchObservedRunningTime="2026-08-05 17:12:58.069277911 +0000 UTC m=+22.768533249"
Aug 05 17:12:58.849362 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:58.849330    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:12:58.849516 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:58.849460    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:12:58.849516 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:58.849503    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:12:58.849638 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:58.849585    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:12:58.893618 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:58.893593    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Aug 05 17:12:58.970668 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:58.970638    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4" event={"ID":"f99723f8-ffe2-4e63-b860-1c1f670994d2","Type":"ContainerStarted","Data":"cddb0c93269062e34d776ad8c1033dede5fa2e21d10f2d7f10de8fe1a1dd19df"}
Aug 05 17:12:58.972182 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:58.972148    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f9xdf" event={"ID":"adeb3a83-c288-4d6e-a4ea-52d6c59b1a83","Type":"ContainerStarted","Data":"29bcc582368ce5f796d91993b6b28e1f31beeacb1c2edbf6f69936e3927aad8d"}
Aug 05 17:12:58.994515 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:58.994435    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f9xdf" podStartSLOduration=5.24927052 podStartE2EDuration="23.994420844s" totalImagesPullingTime="18.745150324s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:12:58.994261236 +0000 UTC m=+23.693516581" watchObservedRunningTime="2026-08-05 17:12:58.994420844 +0000 UTC m=+23.693676488"
Aug 05 17:12:59.720340 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:59.720236    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-08-05T17:12:58.893613288Z","UUID":"b4baf0a9-171a-4a9c-8096-bf0a59480760","Handler":null,"Name":"","Endpoint":""}
Aug 05 17:12:59.721854 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:59.721818    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Aug 05 17:12:59.721961 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:59.721863    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Aug 05 17:12:59.772858 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:59.772828    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:59.773627 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:59.773604    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:59.849327 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:59.849292    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:12:59.849492 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:12:59.849450    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:12:59.979662 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:59.979631    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" event={"ID":"5cab5a3d-0018-4593-8778-0b50c532bd1a","Type":"ContainerStarted","Data":"ac565bd68f32a8270f68715dfa95ce34ced87842364e33b86ffa549bdb82118f"}
Aug 05 17:12:59.979978 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:59.979753    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:12:59.980595 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:12:59.980576    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-85lrg"
Aug 05 17:13:00.849891 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:00.849863    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:00.850108 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:00.849972    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:13:00.850108 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:00.850034    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:00.850222 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:00.850125    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:13:00.983207 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:00.983168    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4" event={"ID":"f99723f8-ffe2-4e63-b860-1c1f670994d2","Type":"ContainerStarted","Data":"0657337ef98adf3ce5047772e16d9639cd8fb8f47ad224ac848780e92a7853da"}
Aug 05 17:13:01.004712 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:01.004669    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2rhm4" podStartSLOduration=4.640585993 podStartE2EDuration="26.004650981s" totalImagesPullingTime="21.364064988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:01.004622317 +0000 UTC m=+25.703877660" watchObservedRunningTime="2026-08-05 17:13:01.004650981 +0000 UTC m=+25.703906300"
Aug 05 17:13:01.852257 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:01.852233    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:01.852436 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:01.852334    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:13:02.849568 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:02.849331    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:02.850263 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:02.849352    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:02.850263 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:02.849605    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:13:02.850263 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:02.849723    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:13:02.988712 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:02.988678    2574 generic.go:356] "Generic (PLEG): container finished" podID="50db4820-71dc-48e9-ba6c-9ddd8f32eb01" containerID="395fd0363f7fc3ae1fd53fc0628e2f28cbad8f5aab289dab7fd31dfd5374f5e1" exitCode=0
Aug 05 17:13:02.988873 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:02.988770    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-phhjj" event={"ID":"50db4820-71dc-48e9-ba6c-9ddd8f32eb01","Type":"ContainerDied","Data":"395fd0363f7fc3ae1fd53fc0628e2f28cbad8f5aab289dab7fd31dfd5374f5e1"}
Aug 05 17:13:02.992307 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:02.992288    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" event={"ID":"5cab5a3d-0018-4593-8778-0b50c532bd1a","Type":"ContainerStarted","Data":"13bf46cb9b37f17ab77b882a01501f884d660b8d7b11d68667ab48afeda9bc6a"}
Aug 05 17:13:02.992556 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:02.992543    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:13:02.992607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:02.992590    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:13:03.007268 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:03.007251    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:13:03.034646 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:03.034601    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp" podStartSLOduration=9.211250754 podStartE2EDuration="28.034588685s" totalImagesPullingTime="18.823337931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:03.03442643 +0000 UTC m=+27.733681771" watchObservedRunningTime="2026-08-05 17:13:03.034588685 +0000 UTC m=+27.733844199"
Aug 05 17:13:03.849710 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:03.849680    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:03.850141 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:03.849861    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:13:03.995997 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:03.995843    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:13:04.014337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:04.014314    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:13:04.406824 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:04.406742    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8gjld"]
Aug 05 17:13:04.406977 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:04.406899    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:04.407059 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:04.407005    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:13:04.409694 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:04.409674    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hp8ck"]
Aug 05 17:13:04.409837 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:04.409760    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:04.409896 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:04.409849    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:13:04.410335 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:04.410310    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zwhvz"]
Aug 05 17:13:04.410430 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:04.410415    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:04.410512 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:04.410495    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:13:04.999741 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:04.999465    2574 generic.go:356] "Generic (PLEG): container finished" podID="50db4820-71dc-48e9-ba6c-9ddd8f32eb01" containerID="ba899db19b9bcdf2739ecfa77c1f1b2c557abaa24cc8d94273b717db35caeea3" exitCode=0
Aug 05 17:13:05.000157 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:04.999560    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-phhjj" event={"ID":"50db4820-71dc-48e9-ba6c-9ddd8f32eb01","Type":"ContainerDied","Data":"ba899db19b9bcdf2739ecfa77c1f1b2c557abaa24cc8d94273b717db35caeea3"}
Aug 05 17:13:05.850215 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:05.850179    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:05.850389 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:05.850264    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:13:05.850389 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:05.850349    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:05.850506 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:05.850452    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:13:06.849846 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:06.849774    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:06.850431 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:06.849870    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:13:07.006072 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:07.006036    2574 generic.go:356] "Generic (PLEG): container finished" podID="50db4820-71dc-48e9-ba6c-9ddd8f32eb01" containerID="77d5a1d6f587d5fe4fba1df2a717c6bf455e6064ff85f71671566816004b4729" exitCode=0
Aug 05 17:13:07.006246 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:07.006090    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-phhjj" event={"ID":"50db4820-71dc-48e9-ba6c-9ddd8f32eb01","Type":"ContainerDied","Data":"77d5a1d6f587d5fe4fba1df2a717c6bf455e6064ff85f71671566816004b4729"}
Aug 05 17:13:07.849504 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:07.849472    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:07.849683 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:07.849475    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:07.849683 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:07.849596    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hp8ck" podUID="25c6e85a-9ed2-446f-9b1f-fa204867ac7a"
Aug 05 17:13:07.849801 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:07.849690    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-8gjld" podUID="de68f860-081a-403c-8a3d-6299868bc425"
Aug 05 17:13:08.849493 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:08.849462    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:08.849890 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:08.849602    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zwhvz" podUID="e3507e66-54c4-4511-ad30-6ec595a04bd4"
Aug 05 17:13:09.418197 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.418172    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-llnmn_cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4/dns-node-resolver/0.log"
Aug 05 17:13:09.513060 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.513027    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:09.513224 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.513081    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:09.513224 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.513152    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:13:09.513224 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.513218    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Aug 05 17:13:09.513374 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.513228    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs podName:e3507e66-54c4-4511-ad30-6ec595a04bd4 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:41.513212665 +0000 UTC m=+66.212467993 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs") pod "network-metrics-daemon-zwhvz" (UID: "e3507e66-54c4-4511-ad30-6ec595a04bd4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Aug 05 17:13:09.513374 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.513268    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret podName:de68f860-081a-403c-8a3d-6299868bc425 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:41.513252587 +0000 UTC m=+66.212507931 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret") pod "global-pull-secret-syncer-8gjld" (UID: "de68f860-081a-403c-8a3d-6299868bc425") : object "kube-system"/"original-pull-secret" not registered
Aug 05 17:13:09.614302 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.614268    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:09.614472 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.614402    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Aug 05 17:13:09.614472 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.614427    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Aug 05 17:13:09.614472 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.614440    2574 projected.go:196] Error preparing data for projected volume kube-api-access-h54dv for pod openshift-network-diagnostics/network-check-target-hp8ck: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:13:09.614633 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.614500    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv podName:25c6e85a-9ed2-446f-9b1f-fa204867ac7a nodeName:}" failed. No retries permitted until 2026-08-05 17:13:41.614482509 +0000 UTC m=+66.313737851 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-h54dv" (UniqueName: "kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv") pod "network-check-target-hp8ck" (UID: "25c6e85a-9ed2-446f-9b1f-fa204867ac7a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Aug 05 17:13:09.646944 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.646923    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-246.ec2.internal" event="NodeReady"
Aug 05 17:13:09.647106 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.647073    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Aug 05 17:13:09.697780 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.697709    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9p8vp"]
Aug 05 17:13:09.700283 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.700261    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n46df"]
Aug 05 17:13:09.700413 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.700394    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.702517 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.702495    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n46df"
Aug 05 17:13:09.702974 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.702951    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Aug 05 17:13:09.703118 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.702963    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Aug 05 17:13:09.703186 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.703104    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mrwwg\""
Aug 05 17:13:09.704956 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.704937    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tvn52\""
Aug 05 17:13:09.704956 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.704952    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Aug 05 17:13:09.705137 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.705007    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Aug 05 17:13:09.705238 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.705223    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Aug 05 17:13:09.713979 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.713931    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n46df"]
Aug 05 17:13:09.714957 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.714935    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9p8vp"]
Aug 05 17:13:09.727143 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.727117    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-shjh5"]
Aug 05 17:13:09.729189 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.729172    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:09.731721 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.731670    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Aug 05 17:13:09.731721 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.731704    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m4zv5\""
Aug 05 17:13:09.731872 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.731839    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Aug 05 17:13:09.731935 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.731897    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Aug 05 17:13:09.732010 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.731989    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Aug 05 17:13:09.739840 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.739781    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-shjh5"]
Aug 05 17:13:09.815578 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.815551    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3968697f-5e9f-4e47-8514-3afa61d501f0-cert\") pod \"ingress-canary-n46df\" (UID: \"3968697f-5e9f-4e47-8514-3afa61d501f0\") " pod="openshift-ingress-canary/ingress-canary-n46df"
Aug 05 17:13:09.815578 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.815586    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a71b652-b075-4a6e-be90-bc33a101e34b-config-volume\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.815783 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.815612    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.815783 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.815687    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29vgr\" (UniqueName: \"kubernetes.io/projected/5a71b652-b075-4a6e-be90-bc33a101e34b-kube-api-access-29vgr\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.815783 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.815722    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a71b652-b075-4a6e-be90-bc33a101e34b-tmp-dir\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.815783 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.815770    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcj7b\" (UniqueName: \"kubernetes.io/projected/3968697f-5e9f-4e47-8514-3afa61d501f0-kube-api-access-dcj7b\") pod \"ingress-canary-n46df\" (UID: \"3968697f-5e9f-4e47-8514-3afa61d501f0\") " pod="openshift-ingress-canary/ingress-canary-n46df"
Aug 05 17:13:09.849981 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.849953    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:09.849981 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.849966    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:09.852933 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.852864    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4r2js\""
Aug 05 17:13:09.852933 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.852904    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 05 17:13:09.852933 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.852906    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:13:09.853301 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.853276    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 05 17:13:09.916215 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcj7b\" (UniqueName: \"kubernetes.io/projected/3968697f-5e9f-4e47-8514-3afa61d501f0-kube-api-access-dcj7b\") pod \"ingress-canary-n46df\" (UID: \"3968697f-5e9f-4e47-8514-3afa61d501f0\") " pod="openshift-ingress-canary/ingress-canary-n46df"
Aug 05 17:13:09.916353 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916268    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:09.916353 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916298    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3968697f-5e9f-4e47-8514-3afa61d501f0-cert\") pod \"ingress-canary-n46df\" (UID: \"3968697f-5e9f-4e47-8514-3afa61d501f0\") " pod="openshift-ingress-canary/ingress-canary-n46df"
Aug 05 17:13:09.916353 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916325    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfdbv\" (UniqueName: \"kubernetes.io/projected/731c2e93-915e-4456-8eef-c862ced23934-kube-api-access-dfdbv\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:09.916570 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916381    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a71b652-b075-4a6e-be90-bc33a101e34b-config-volume\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.916570 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916412    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/731c2e93-915e-4456-8eef-c862ced23934-crio-socket\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:09.916570 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916436    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:09.916570 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916491    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.916570 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916539    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29vgr\" (UniqueName: \"kubernetes.io/projected/5a71b652-b075-4a6e-be90-bc33a101e34b-kube-api-access-29vgr\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.916570 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916568    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/731c2e93-915e-4456-8eef-c862ced23934-data-volume\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:09.916790 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.916580    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:13:09.916790 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916606    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a71b652-b075-4a6e-be90-bc33a101e34b-tmp-dir\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.916790 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:09.916633    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls podName:5a71b652-b075-4a6e-be90-bc33a101e34b nodeName:}" failed. No retries permitted until 2026-08-05 17:13:10.416615778 +0000 UTC m=+35.115871102 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls") pod "dns-default-9p8vp" (UID: "5a71b652-b075-4a6e-be90-bc33a101e34b") : secret "dns-default-metrics-tls" not found
Aug 05 17:13:09.916891 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916860    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a71b652-b075-4a6e-be90-bc33a101e34b-tmp-dir\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.916976 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.916957    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5a71b652-b075-4a6e-be90-bc33a101e34b-config-volume\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:09.920760 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.920741    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3968697f-5e9f-4e47-8514-3afa61d501f0-cert\") pod \"ingress-canary-n46df\" (UID: \"3968697f-5e9f-4e47-8514-3afa61d501f0\") " pod="openshift-ingress-canary/ingress-canary-n46df"
Aug 05 17:13:09.925632 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.925597    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcj7b\" (UniqueName: \"kubernetes.io/projected/3968697f-5e9f-4e47-8514-3afa61d501f0-kube-api-access-dcj7b\") pod \"ingress-canary-n46df\" (UID: \"3968697f-5e9f-4e47-8514-3afa61d501f0\") " pod="openshift-ingress-canary/ingress-canary-n46df"
Aug 05 17:13:09.925752 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:09.925604    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29vgr\" (UniqueName: \"kubernetes.io/projected/5a71b652-b075-4a6e-be90-bc33a101e34b-kube-api-access-29vgr\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:10.017512 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.017426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/731c2e93-915e-4456-8eef-c862ced23934-data-volume\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.017667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.017531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.017667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.017561    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdbv\" (UniqueName: \"kubernetes.io/projected/731c2e93-915e-4456-8eef-c862ced23934-kube-api-access-dfdbv\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.017667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.017596    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/731c2e93-915e-4456-8eef-c862ced23934-crio-socket\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.017667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.017622    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.017872 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.017679    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:13:10.017872 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.017723    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:10.017872 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.017752    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm podName:731c2e93-915e-4456-8eef-c862ced23934 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:10.517731049 +0000 UTC m=+35.216986388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shjh5" (UID: "731c2e93-915e-4456-8eef-c862ced23934") : configmap "kube-rbac-proxy" not found
Aug 05 17:13:10.017872 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.017771    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls podName:731c2e93-915e-4456-8eef-c862ced23934 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:10.517762728 +0000 UTC m=+35.217018053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shjh5" (UID: "731c2e93-915e-4456-8eef-c862ced23934") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:10.017872 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.017789    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/731c2e93-915e-4456-8eef-c862ced23934-data-volume\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.018139 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.017897    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/731c2e93-915e-4456-8eef-c862ced23934-crio-socket\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.020394 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.020373    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n46df"
Aug 05 17:13:10.030544 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.030525    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfdbv\" (UniqueName: \"kubernetes.io/projected/731c2e93-915e-4456-8eef-c862ced23934-kube-api-access-dfdbv\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.402833 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.402764    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s5rqv_975a4dd9-7be5-40da-b2ce-ab8162b1d289/node-ca/0.log"
Aug 05 17:13:10.420542 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.420518    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:10.420688 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.420671    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:13:10.420755 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.420741    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls podName:5a71b652-b075-4a6e-be90-bc33a101e34b nodeName:}" failed. No retries permitted until 2026-08-05 17:13:11.420720978 +0000 UTC m=+36.119976312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls") pod "dns-default-9p8vp" (UID: "5a71b652-b075-4a6e-be90-bc33a101e34b") : secret "dns-default-metrics-tls" not found
Aug 05 17:13:10.521089 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.521033    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.521089 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.521100    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:13:10.521339 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.521135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:10.521339 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.521190    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm podName:731c2e93-915e-4456-8eef-c862ced23934 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:11.521161715 +0000 UTC m=+36.220417047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shjh5" (UID: "731c2e93-915e-4456-8eef-c862ced23934") : configmap "kube-rbac-proxy" not found
Aug 05 17:13:10.521339 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.521250    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:10.521339 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:10.521307    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls podName:731c2e93-915e-4456-8eef-c862ced23934 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:11.521292622 +0000 UTC m=+36.220547943 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shjh5" (UID: "731c2e93-915e-4456-8eef-c862ced23934") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:10.799576 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.799497    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-pkthc"]
Aug 05 17:13:10.801705 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.801688    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:10.804157 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.804135    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Aug 05 17:13:10.804399 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.804384    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Aug 05 17:13:10.805674 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.805651    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Aug 05 17:13:10.806234 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.806216    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-tnf86\""
Aug 05 17:13:10.806534 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.806515    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Aug 05 17:13:10.814856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.814833    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-pkthc"]
Aug 05 17:13:10.848922 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.848896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:10.851641 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.851617    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:13:10.851979 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.851645    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vwprk\""
Aug 05 17:13:10.924478 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.924295    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/39fa1414-ae70-457f-963a-93a0a7dcebe2-signing-key\") pod \"service-ca-98c88fbb5-pkthc\" (UID: \"39fa1414-ae70-457f-963a-93a0a7dcebe2\") " pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:10.924632 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.924512    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/39fa1414-ae70-457f-963a-93a0a7dcebe2-signing-cabundle\") pod \"service-ca-98c88fbb5-pkthc\" (UID: \"39fa1414-ae70-457f-963a-93a0a7dcebe2\") " pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:10.924632 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:10.924562    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gvlt\" (UniqueName: \"kubernetes.io/projected/39fa1414-ae70-457f-963a-93a0a7dcebe2-kube-api-access-2gvlt\") pod \"service-ca-98c88fbb5-pkthc\" (UID: \"39fa1414-ae70-457f-963a-93a0a7dcebe2\") " pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:11.019081 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.019056    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n46df"]
Aug 05 17:13:11.022468 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:11.022439    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3968697f_5e9f_4e47_8514_3afa61d501f0.slice/crio-74f61ef9937d528e145aee28f9ef7672922f0d100c5f00495fee5b45f6ed2222 WatchSource:0}: Error finding container 74f61ef9937d528e145aee28f9ef7672922f0d100c5f00495fee5b45f6ed2222: Status 404 returned error can't find the container with id 74f61ef9937d528e145aee28f9ef7672922f0d100c5f00495fee5b45f6ed2222
Aug 05 17:13:11.024979 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.024958    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/39fa1414-ae70-457f-963a-93a0a7dcebe2-signing-cabundle\") pod \"service-ca-98c88fbb5-pkthc\" (UID: \"39fa1414-ae70-457f-963a-93a0a7dcebe2\") " pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:11.025654 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.025042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvlt\" (UniqueName: \"kubernetes.io/projected/39fa1414-ae70-457f-963a-93a0a7dcebe2-kube-api-access-2gvlt\") pod \"service-ca-98c88fbb5-pkthc\" (UID: \"39fa1414-ae70-457f-963a-93a0a7dcebe2\") " pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:11.025654 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.025222    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/39fa1414-ae70-457f-963a-93a0a7dcebe2-signing-key\") pod \"service-ca-98c88fbb5-pkthc\" (UID: \"39fa1414-ae70-457f-963a-93a0a7dcebe2\") " pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:11.025844 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.025727    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/39fa1414-ae70-457f-963a-93a0a7dcebe2-signing-cabundle\") pod \"service-ca-98c88fbb5-pkthc\" (UID: \"39fa1414-ae70-457f-963a-93a0a7dcebe2\") " pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:11.028730 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.028700    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/39fa1414-ae70-457f-963a-93a0a7dcebe2-signing-key\") pod \"service-ca-98c88fbb5-pkthc\" (UID: \"39fa1414-ae70-457f-963a-93a0a7dcebe2\") " pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:11.034717 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.034691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gvlt\" (UniqueName: \"kubernetes.io/projected/39fa1414-ae70-457f-963a-93a0a7dcebe2-kube-api-access-2gvlt\") pod \"service-ca-98c88fbb5-pkthc\" (UID: \"39fa1414-ae70-457f-963a-93a0a7dcebe2\") " pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:11.114864 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.114841    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-98c88fbb5-pkthc"
Aug 05 17:13:11.249510 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.249480    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-98c88fbb5-pkthc"]
Aug 05 17:13:11.252641 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:11.252613    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39fa1414_ae70_457f_963a_93a0a7dcebe2.slice/crio-775e004d7d349ace28652198bf0c9ba88f98504debe1caf7396532a587ded838 WatchSource:0}: Error finding container 775e004d7d349ace28652198bf0c9ba88f98504debe1caf7396532a587ded838: Status 404 returned error can't find the container with id 775e004d7d349ace28652198bf0c9ba88f98504debe1caf7396532a587ded838
Aug 05 17:13:11.427537 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.427469    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:11.427682 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:11.427603    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:13:11.427682 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:11.427662    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls podName:5a71b652-b075-4a6e-be90-bc33a101e34b nodeName:}" failed. No retries permitted until 2026-08-05 17:13:13.427648227 +0000 UTC m=+38.126903547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls") pod "dns-default-9p8vp" (UID: "5a71b652-b075-4a6e-be90-bc33a101e34b") : secret "dns-default-metrics-tls" not found
Aug 05 17:13:11.528427 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.528394    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:11.528589 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:11.528452    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:11.528589 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:11.528500    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Aug 05 17:13:11.528589 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:11.528561    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:11.528589 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:11.528584    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm podName:731c2e93-915e-4456-8eef-c862ced23934 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:13.528559986 +0000 UTC m=+38.227815309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm") pod "insights-runtime-extractor-shjh5" (UID: "731c2e93-915e-4456-8eef-c862ced23934") : configmap "kube-rbac-proxy" not found
Aug 05 17:13:11.528810 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:11.528607    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls podName:731c2e93-915e-4456-8eef-c862ced23934 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:13.528595752 +0000 UTC m=+38.227851072 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shjh5" (UID: "731c2e93-915e-4456-8eef-c862ced23934") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:12.020935 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:12.020612    2574 generic.go:356] "Generic (PLEG): container finished" podID="50db4820-71dc-48e9-ba6c-9ddd8f32eb01" containerID="55bb66ad353fdb9dce732974d1b840be5bab3572d7111372d14d736e5e4c162c" exitCode=0
Aug 05 17:13:12.020935 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:12.020698    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-phhjj" event={"ID":"50db4820-71dc-48e9-ba6c-9ddd8f32eb01","Type":"ContainerDied","Data":"55bb66ad353fdb9dce732974d1b840be5bab3572d7111372d14d736e5e4c162c"}
Aug 05 17:13:12.022762 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:12.022728    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-pkthc" event={"ID":"39fa1414-ae70-457f-963a-93a0a7dcebe2","Type":"ContainerStarted","Data":"775e004d7d349ace28652198bf0c9ba88f98504debe1caf7396532a587ded838"}
Aug 05 17:13:12.024619 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:12.024593    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n46df" event={"ID":"3968697f-5e9f-4e47-8514-3afa61d501f0","Type":"ContainerStarted","Data":"74f61ef9937d528e145aee28f9ef7672922f0d100c5f00495fee5b45f6ed2222"}
Aug 05 17:13:13.030516 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:13.030467    2574 generic.go:356] "Generic (PLEG): container finished" podID="50db4820-71dc-48e9-ba6c-9ddd8f32eb01" containerID="658eb996e1fa04b42c286d4e22d60c4e9730ca779b22da9b3fbb95049c119de4" exitCode=0
Aug 05 17:13:13.030516 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:13.030516    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-phhjj" event={"ID":"50db4820-71dc-48e9-ba6c-9ddd8f32eb01","Type":"ContainerDied","Data":"658eb996e1fa04b42c286d4e22d60c4e9730ca779b22da9b3fbb95049c119de4"}
Aug 05 17:13:13.442512 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:13.442440    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:13.442645 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:13.442590    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:13:13.442680 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:13.442651    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls podName:5a71b652-b075-4a6e-be90-bc33a101e34b nodeName:}" failed. No retries permitted until 2026-08-05 17:13:17.442634725 +0000 UTC m=+42.141890059 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls") pod "dns-default-9p8vp" (UID: "5a71b652-b075-4a6e-be90-bc33a101e34b") : secret "dns-default-metrics-tls" not found
Aug 05 17:13:13.543334 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:13.543301    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:13.543488 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:13.543371    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:13.543555 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:13.543520    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:13.543604 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:13.543592    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls podName:731c2e93-915e-4456-8eef-c862ced23934 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:17.543567183 +0000 UTC m=+42.242822503 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shjh5" (UID: "731c2e93-915e-4456-8eef-c862ced23934") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:13.550302 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:13.550281    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/731c2e93-915e-4456-8eef-c862ced23934-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:15.038837 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:15.038800    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-phhjj" event={"ID":"50db4820-71dc-48e9-ba6c-9ddd8f32eb01","Type":"ContainerStarted","Data":"b40a634c7e485b31da065926d717d6aefcc19e33a9deac3b03242f0eb4ce3c91"}
Aug 05 17:13:15.040265 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:15.040239    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-98c88fbb5-pkthc" event={"ID":"39fa1414-ae70-457f-963a-93a0a7dcebe2","Type":"ContainerStarted","Data":"a859d4ec684ac31ecab0a4ba0f4ccb2a5092b2ad684ea1f2a6414645a5c6c422"}
Aug 05 17:13:15.041462 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:15.041441    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n46df" event={"ID":"3968697f-5e9f-4e47-8514-3afa61d501f0","Type":"ContainerStarted","Data":"3d5f36fe2634b55ad40f2848dcd2424daf7b0c4b50981ffa6e700d57f041b586"}
Aug 05 17:13:15.075295 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:15.075260    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-phhjj" podStartSLOduration=10.829265381 podStartE2EDuration="40.075249398s" totalImagesPullingTime="29.245984017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:15.074138627 +0000 UTC m=+39.773393969" watchObservedRunningTime="2026-08-05 17:13:15.075249398 +0000 UTC m=+39.774504743"
Aug 05 17:13:15.099822 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:15.099784    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n46df" podStartSLOduration=2.981443975 podStartE2EDuration="6.099772003s" totalImagesPullingTime="3.118328028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:15.098754603 +0000 UTC m=+39.798009945" watchObservedRunningTime="2026-08-05 17:13:15.099772003 +0000 UTC m=+39.799027345"
Aug 05 17:13:15.141137 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:15.141098    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-98c88fbb5-pkthc" podStartSLOduration=2.249144568 podStartE2EDuration="5.141084321s" totalImagesPullingTime="2.891939753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:15.140651336 +0000 UTC m=+39.839906679" watchObservedRunningTime="2026-08-05 17:13:15.141084321 +0000 UTC m=+39.840339663"
Aug 05 17:13:17.476156 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:17.476124    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:17.476512 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:17.476282    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Aug 05 17:13:17.476512 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:17.476343    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls podName:5a71b652-b075-4a6e-be90-bc33a101e34b nodeName:}" failed. No retries permitted until 2026-08-05 17:13:25.4763272 +0000 UTC m=+50.175582520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls") pod "dns-default-9p8vp" (UID: "5a71b652-b075-4a6e-be90-bc33a101e34b") : secret "dns-default-metrics-tls" not found
Aug 05 17:13:17.576522 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:17.576493    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:17.576665 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:17.576633    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:17.576705 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:17.576690    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls podName:731c2e93-915e-4456-8eef-c862ced23934 nodeName:}" failed. No retries permitted until 2026-08-05 17:13:25.576674124 +0000 UTC m=+50.275929443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls") pod "insights-runtime-extractor-shjh5" (UID: "731c2e93-915e-4456-8eef-c862ced23934") : secret "insights-runtime-extractor-tls" not found
Aug 05 17:13:25.530552 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:25.530515    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:25.533826 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:25.533805    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5a71b652-b075-4a6e-be90-bc33a101e34b-metrics-tls\") pod \"dns-default-9p8vp\" (UID: \"5a71b652-b075-4a6e-be90-bc33a101e34b\") " pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:25.613368 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:25.613340    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:25.631779 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:25.631743    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:25.633836 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:25.633819    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/731c2e93-915e-4456-8eef-c862ced23934-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-shjh5\" (UID: \"731c2e93-915e-4456-8eef-c862ced23934\") " pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:25.638725 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:25.638709    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-shjh5"
Aug 05 17:13:25.786682 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:25.786550    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9p8vp"]
Aug 05 17:13:25.791498 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:25.791466    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a71b652_b075_4a6e_be90_bc33a101e34b.slice/crio-0f156b15aecc780eefa07a9091c1e3012e6de9c4877ef67f1992dc7037e0ba5e WatchSource:0}: Error finding container 0f156b15aecc780eefa07a9091c1e3012e6de9c4877ef67f1992dc7037e0ba5e: Status 404 returned error can't find the container with id 0f156b15aecc780eefa07a9091c1e3012e6de9c4877ef67f1992dc7037e0ba5e
Aug 05 17:13:25.807623 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:25.807598    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-shjh5"]
Aug 05 17:13:25.810571 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:25.810550    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod731c2e93_915e_4456_8eef_c862ced23934.slice/crio-e4186891fba45111b62b4d6e1879a474dc886aa3605c6bc0603597c983c86941 WatchSource:0}: Error finding container e4186891fba45111b62b4d6e1879a474dc886aa3605c6bc0603597c983c86941: Status 404 returned error can't find the container with id e4186891fba45111b62b4d6e1879a474dc886aa3605c6bc0603597c983c86941
Aug 05 17:13:26.069504 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:26.069425    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shjh5" event={"ID":"731c2e93-915e-4456-8eef-c862ced23934","Type":"ContainerStarted","Data":"fda0d7bcc633eb4b4f992e2d38181fefe752f5f27c4df17a818df43fffff5dbc"}
Aug 05 17:13:26.069504 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:26.069483    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shjh5" event={"ID":"731c2e93-915e-4456-8eef-c862ced23934","Type":"ContainerStarted","Data":"e4186891fba45111b62b4d6e1879a474dc886aa3605c6bc0603597c983c86941"}
Aug 05 17:13:26.070684 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:26.070656    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9p8vp" event={"ID":"5a71b652-b075-4a6e-be90-bc33a101e34b","Type":"ContainerStarted","Data":"0f156b15aecc780eefa07a9091c1e3012e6de9c4877ef67f1992dc7037e0ba5e"}
Aug 05 17:13:28.079567 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:28.079520    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9p8vp" event={"ID":"5a71b652-b075-4a6e-be90-bc33a101e34b","Type":"ContainerStarted","Data":"91217691dca7d9e6404d30918c89b658c10937dcc458f979c996e8042fdd7207"}
Aug 05 17:13:28.080086 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:28.079629    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9p8vp" event={"ID":"5a71b652-b075-4a6e-be90-bc33a101e34b","Type":"ContainerStarted","Data":"cc8afffa1d599b998c9c4f8b2140c928ed342e581ba0a3c39c06d3a8e6b8193a"}
Aug 05 17:13:28.080086 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:28.079665    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:28.081333 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:28.081306    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shjh5" event={"ID":"731c2e93-915e-4456-8eef-c862ced23934","Type":"ContainerStarted","Data":"94c7bb9812388af4290f81d3c9a62dc0a196224754541c7bca4162ace07689d6"}
Aug 05 17:13:28.101557 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:28.101514    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9p8vp" podStartSLOduration=17.255739036 podStartE2EDuration="19.101497417s" totalImagesPullingTime="1.845758381s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:28.100353699 +0000 UTC m=+52.799609044" watchObservedRunningTime="2026-08-05 17:13:28.101497417 +0000 UTC m=+52.800752776"
Aug 05 17:13:29.085847 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:29.085815    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-shjh5" event={"ID":"731c2e93-915e-4456-8eef-c862ced23934","Type":"ContainerStarted","Data":"71e40ccc26b1d117c4538f53f4808e7b993c405ebaae98cb859b3ff729c02e7e"}
Aug 05 17:13:29.106670 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:29.106626    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-shjh5" podStartSLOduration=17.212556601 podStartE2EDuration="20.106610435s" totalImagesPullingTime="2.894053834s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:13:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:29.105389498 +0000 UTC m=+53.804644831" watchObservedRunningTime="2026-08-05 17:13:29.106610435 +0000 UTC m=+53.805865795"
Aug 05 17:13:34.883095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.883065    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"]
Aug 05 17:13:34.915364 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.915334    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"]
Aug 05 17:13:34.915515 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.915470    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:34.917877 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.917853    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Aug 05 17:13:34.918718 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.918699    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Aug 05 17:13:34.918854 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.918840    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Aug 05 17:13:34.919086 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.919067    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Aug 05 17:13:34.919188 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.919097    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Aug 05 17:13:34.919188 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.919125    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Aug 05 17:13:34.919188 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.919140    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Aug 05 17:13:34.929501 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.929482    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"]
Aug 05 17:13:34.929581 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.929506    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"]
Aug 05 17:13:34.929581 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.929565    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"
Aug 05 17:13:34.931870 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.931842    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-lwcdr\""
Aug 05 17:13:34.932007 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.931992    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Aug 05 17:13:34.995830 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.995800    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:34.995931 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.995834    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-595b88948b-knsd9\" (UID: \"e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"
Aug 05 17:13:34.995931 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.995852    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:34.995931 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.995887    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d970cc-528b-4ed3-a554-73bf54dd7cd7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:34.996062 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.995943    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jscsp\" (UniqueName: \"kubernetes.io/projected/47d970cc-528b-4ed3-a554-73bf54dd7cd7-kube-api-access-jscsp\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:34.996062 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.996046    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-ca\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:34.996126 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.996067    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj88g\" (UniqueName: \"kubernetes.io/projected/e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d-kube-api-access-zj88g\") pod \"managed-serviceaccount-addon-agent-595b88948b-knsd9\" (UID: \"e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"
Aug 05 17:13:34.996126 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:34.996102    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-hub\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.096959 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.096934    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-ca\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.097099 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.096964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj88g\" (UniqueName: \"kubernetes.io/projected/e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d-kube-api-access-zj88g\") pod \"managed-serviceaccount-addon-agent-595b88948b-knsd9\" (UID: \"e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"
Aug 05 17:13:35.097099 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.096990    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-hub\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.097099 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.097035    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.097099 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.097064    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-595b88948b-knsd9\" (UID: \"e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"
Aug 05 17:13:35.097099 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.097088    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.097357 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.097126    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d970cc-528b-4ed3-a554-73bf54dd7cd7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.097357 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.097155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jscsp\" (UniqueName: \"kubernetes.io/projected/47d970cc-528b-4ed3-a554-73bf54dd7cd7-kube-api-access-jscsp\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.098155 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.098130    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/47d970cc-528b-4ed3-a554-73bf54dd7cd7-additional-service-ca\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.099600 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.099572    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-hub\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.099695 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.099572    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.099909 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.099886    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.099971 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.099909    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/47d970cc-528b-4ed3-a554-73bf54dd7cd7-ca\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.100100 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.100078    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-595b88948b-knsd9\" (UID: \"e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"
Aug 05 17:13:35.117663 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.117632    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj88g\" (UniqueName: \"kubernetes.io/projected/e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d-kube-api-access-zj88g\") pod \"managed-serviceaccount-addon-agent-595b88948b-knsd9\" (UID: \"e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"
Aug 05 17:13:35.118479 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.118459    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jscsp\" (UniqueName: \"kubernetes.io/projected/47d970cc-528b-4ed3-a554-73bf54dd7cd7-kube-api-access-jscsp\") pod \"cluster-proxy-proxy-agent-78864dc6bf-zjdp2\" (UID: \"47d970cc-528b-4ed3-a554-73bf54dd7cd7\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.237995 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.237965    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"
Aug 05 17:13:35.244092 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.244075    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"
Aug 05 17:13:35.401444 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.401418    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2"]
Aug 05 17:13:35.404242 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:35.404213    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47d970cc_528b_4ed3_a554_73bf54dd7cd7.slice/crio-9d9cabacc470d7f4d381689a6960584677ab96df5aa3fa83e433f3126fa4c329 WatchSource:0}: Error finding container 9d9cabacc470d7f4d381689a6960584677ab96df5aa3fa83e433f3126fa4c329: Status 404 returned error can't find the container with id 9d9cabacc470d7f4d381689a6960584677ab96df5aa3fa83e433f3126fa4c329
Aug 05 17:13:35.432858 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:35.432835    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9"]
Aug 05 17:13:35.435558 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:35.435531    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8c8b3b7_bb5e_439b_aeb1_b9248b7bc24d.slice/crio-73e49dab69b4581001948061b8459eca3503b5b638a48f4fe3a88e093937142d WatchSource:0}: Error finding container 73e49dab69b4581001948061b8459eca3503b5b638a48f4fe3a88e093937142d: Status 404 returned error can't find the container with id 73e49dab69b4581001948061b8459eca3503b5b638a48f4fe3a88e093937142d
Aug 05 17:13:36.013208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:36.013182    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5bfsp"
Aug 05 17:13:36.106361 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:36.106304    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2" event={"ID":"47d970cc-528b-4ed3-a554-73bf54dd7cd7","Type":"ContainerStarted","Data":"9d9cabacc470d7f4d381689a6960584677ab96df5aa3fa83e433f3126fa4c329"}
Aug 05 17:13:36.107653 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:36.107629    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9" event={"ID":"e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d","Type":"ContainerStarted","Data":"73e49dab69b4581001948061b8459eca3503b5b638a48f4fe3a88e093937142d"}
Aug 05 17:13:38.088436 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:38.088408    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9p8vp"
Aug 05 17:13:41.550932 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.550895    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:41.551371 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.550947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:41.553530 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.553508    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Aug 05 17:13:41.554167 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.554147    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Aug 05 17:13:41.564239 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.564216    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3507e66-54c4-4511-ad30-6ec595a04bd4-metrics-certs\") pod \"network-metrics-daemon-zwhvz\" (UID: \"e3507e66-54c4-4511-ad30-6ec595a04bd4\") " pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:41.564349 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.564223    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/de68f860-081a-403c-8a3d-6299868bc425-original-pull-secret\") pod \"global-pull-secret-syncer-8gjld\" (UID: \"de68f860-081a-403c-8a3d-6299868bc425\") " pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:41.651920 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.651887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:41.654506 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.654480    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Aug 05 17:13:41.661998 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.661978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-8gjld"
Aug 05 17:13:41.665422 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.665402    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Aug 05 17:13:41.676234 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.676211    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h54dv\" (UniqueName: \"kubernetes.io/projected/25c6e85a-9ed2-446f-9b1f-fa204867ac7a-kube-api-access-h54dv\") pod \"network-check-target-hp8ck\" (UID: \"25c6e85a-9ed2-446f-9b1f-fa204867ac7a\") " pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:41.761334 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.761304    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vwprk\""
Aug 05 17:13:41.769361 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.769338    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zwhvz"
Aug 05 17:13:41.970115 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.970077    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4r2js\""
Aug 05 17:13:41.978168 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:41.978139    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:42.951843 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:42.951817    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zwhvz"]
Aug 05 17:13:42.955125 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:42.955101    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3507e66_54c4_4511_ad30_6ec595a04bd4.slice/crio-2d8de17ada54517c4401bf6d0f96e8d914af0b8eba39f5c8ad1de3bb23bd5f71 WatchSource:0}: Error finding container 2d8de17ada54517c4401bf6d0f96e8d914af0b8eba39f5c8ad1de3bb23bd5f71: Status 404 returned error can't find the container with id 2d8de17ada54517c4401bf6d0f96e8d914af0b8eba39f5c8ad1de3bb23bd5f71
Aug 05 17:13:43.130823 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:43.130743    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2" event={"ID":"47d970cc-528b-4ed3-a554-73bf54dd7cd7","Type":"ContainerStarted","Data":"679c1a3721ea5b589b280b9b64db08a535df40b0fbb239a0c660542492dfd17d"}
Aug 05 17:13:43.131786 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:43.131765    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zwhvz" event={"ID":"e3507e66-54c4-4511-ad30-6ec595a04bd4","Type":"ContainerStarted","Data":"2d8de17ada54517c4401bf6d0f96e8d914af0b8eba39f5c8ad1de3bb23bd5f71"}
Aug 05 17:13:43.175720 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:43.175689    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hp8ck"]
Aug 05 17:13:43.178835 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:43.178813    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-8gjld"]
Aug 05 17:13:43.217028 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:43.216979    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c6e85a_9ed2_446f_9b1f_fa204867ac7a.slice/crio-db00b978265de622b744bf573ee91d35100b41b3b1c55120a42e9c20280e83b8 WatchSource:0}: Error finding container db00b978265de622b744bf573ee91d35100b41b3b1c55120a42e9c20280e83b8: Status 404 returned error can't find the container with id db00b978265de622b744bf573ee91d35100b41b3b1c55120a42e9c20280e83b8
Aug 05 17:13:43.217753 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:43.217728    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde68f860_081a_403c_8a3d_6299868bc425.slice/crio-c69af5bb81f753cc65c95e815e5758866d2fd5147446dd8a4d59b7b335155fe6 WatchSource:0}: Error finding container c69af5bb81f753cc65c95e815e5758866d2fd5147446dd8a4d59b7b335155fe6: Status 404 returned error can't find the container with id c69af5bb81f753cc65c95e815e5758866d2fd5147446dd8a4d59b7b335155fe6
Aug 05 17:13:44.137718 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:44.137673    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2" event={"ID":"47d970cc-528b-4ed3-a554-73bf54dd7cd7","Type":"ContainerStarted","Data":"e18b24bd9c88e0f2b189b14c56df6d816e2f826cf728f1db72fb2753fe87abf6"}
Aug 05 17:13:44.138185 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:44.137728    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2" event={"ID":"47d970cc-528b-4ed3-a554-73bf54dd7cd7","Type":"ContainerStarted","Data":"f3d2754056a02a22dabae9b9cf979895d3448c70a88e1258d15cf0cc96d87fce"}
Aug 05 17:13:44.139383 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:44.139353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9" event={"ID":"e8c8b3b7-bb5e-439b-aeb1-b9248b7bc24d","Type":"ContainerStarted","Data":"4c3693c35e0009aa7caeb051fab3c9d42b86af86df5e445ce33bb8342860c3d0"}
Aug 05 17:13:44.140568 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:44.140543    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8gjld" event={"ID":"de68f860-081a-403c-8a3d-6299868bc425","Type":"ContainerStarted","Data":"c69af5bb81f753cc65c95e815e5758866d2fd5147446dd8a4d59b7b335155fe6"}
Aug 05 17:13:44.141894 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:44.141868    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hp8ck" event={"ID":"25c6e85a-9ed2-446f-9b1f-fa204867ac7a","Type":"ContainerStarted","Data":"db00b978265de622b744bf573ee91d35100b41b3b1c55120a42e9c20280e83b8"}
Aug 05 17:13:44.157054 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:44.156990    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78864dc6bf-zjdp2" podStartSLOduration=2.748566134 podStartE2EDuration="10.156973985s" totalImagesPullingTime="7.408407851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:44.155773768 +0000 UTC m=+68.855029111" watchObservedRunningTime="2026-08-05 17:13:44.156973985 +0000 UTC m=+68.856229333"
Aug 05 17:13:44.196904 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:44.196657    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-595b88948b-knsd9" podStartSLOduration=2.381526873 podStartE2EDuration="10.196638261s" totalImagesPullingTime="7.815111388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:44.196040215 +0000 UTC m=+68.895295554" watchObservedRunningTime="2026-08-05 17:13:44.196638261 +0000 UTC m=+68.895893603"
Aug 05 17:13:45.150035 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.149940    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zwhvz" event={"ID":"e3507e66-54c4-4511-ad30-6ec595a04bd4","Type":"ContainerStarted","Data":"786f3c8a12ce50b7bb198f7dd6ea824b0a264a67666757fac3090d8dff6ffeca"}
Aug 05 17:13:45.150035 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.149998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zwhvz" event={"ID":"e3507e66-54c4-4511-ad30-6ec595a04bd4","Type":"ContainerStarted","Data":"5295e9f2d8184c9f2c4d718f78a59fb44bb6cf8d3773fdde796fea648c1e383d"}
Aug 05 17:13:45.199196 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.199132    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zwhvz" podStartSLOduration=68.737661746 podStartE2EDuration="1m10.199110162s" totalImagesPullingTime="1.461448416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:45.176722247 +0000 UTC m=+69.875977590" watchObservedRunningTime="2026-08-05 17:13:45.199110162 +0000 UTC m=+69.898365511"
Aug 05 17:13:45.199749 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.199727    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zptlm"]
Aug 05 17:13:45.222175 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.222150    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.224303 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.224280    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Aug 05 17:13:45.224802 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.224781    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Aug 05 17:13:45.225216 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.225063    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Aug 05 17:13:45.225216 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.225123    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Aug 05 17:13:45.225401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.225279    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Aug 05 17:13:45.225401 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.225343    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lzm78\""
Aug 05 17:13:45.225742 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.225618    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Aug 05 17:13:45.282490 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.282098    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2pqj\" (UniqueName: \"kubernetes.io/projected/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-kube-api-access-l2pqj\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.282490 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.282136    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-textfile\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.282490 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.282177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.282490 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.282200    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-tls\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.282490 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.282230    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-wtmp\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.282490 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.282271    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-root\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.282490 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.282298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.282490 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.282326    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-metrics-client-ca\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.282490 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.282392    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-sys\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.383300    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-sys\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.383347    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2pqj\" (UniqueName: \"kubernetes.io/projected/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-kube-api-access-l2pqj\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.383376    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-textfile\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.383410    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.383433    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-tls\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.383461    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-wtmp\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.383488    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-root\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.383516    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.383542    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-metrics-client-ca\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.384094    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-sys\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.384190    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-metrics-client-ca\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.384240    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-wtmp\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.384292 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.384259    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-textfile\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.385047 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.384707    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-accelerators-collector-config\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.385047 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.385033    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-root\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.390761 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.388129    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-tls\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.391075 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.391045    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.404424 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.404365    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2pqj\" (UniqueName: \"kubernetes.io/projected/e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb-kube-api-access-l2pqj\") pod \"node-exporter-zptlm\" (UID: \"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb\") " pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.545389 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:45.545139    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zptlm"
Aug 05 17:13:45.570471 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:45.570438    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3cbb0d3_c038_4b4d_b5bf_8c32e32ff5cb.slice/crio-833afbaaaad23185225e10caf088d4f8b0707bbe22010b5baaf5a93fe562a215 WatchSource:0}: Error finding container 833afbaaaad23185225e10caf088d4f8b0707bbe22010b5baaf5a93fe562a215: Status 404 returned error can't find the container with id 833afbaaaad23185225e10caf088d4f8b0707bbe22010b5baaf5a93fe562a215
Aug 05 17:13:46.162322 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.162285    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zptlm" event={"ID":"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb","Type":"ContainerStarted","Data":"833afbaaaad23185225e10caf088d4f8b0707bbe22010b5baaf5a93fe562a215"}
Aug 05 17:13:46.358291 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.358255    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:13:46.361228 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.361193    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.364049 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.364000    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Aug 05 17:13:46.365182 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.365070    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Aug 05 17:13:46.365182 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.365085    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Aug 05 17:13:46.365363 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.365265    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Aug 05 17:13:46.365419 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.365384    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Aug 05 17:13:46.365519 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.365417    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Aug 05 17:13:46.365519 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.365425    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-z7sqb\""
Aug 05 17:13:46.367192 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.367156    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Aug 05 17:13:46.367307 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.367231    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Aug 05 17:13:46.367307 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.367286    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Aug 05 17:13:46.384796 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.384478    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:13:46.492486 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492452    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492576    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492612    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492643    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492666    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492912 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492689    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492912 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492708    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492912 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sscjm\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-kube-api-access-sscjm\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492912 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492776    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-out\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492912 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492812    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492912 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-web-config\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.492912 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.492866    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.593958 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.593925    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594145 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.593972    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594145 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594095    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594145 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594132    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594311 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594160    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594311 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594311 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594209    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594311 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594237    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sscjm\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-kube-api-access-sscjm\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594311 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594270    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-out\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594311 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594295    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594565 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594324    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-web-config\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594565 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594351    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.594565 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.594386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.595312 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.595286    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.595427 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.595380    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.597639 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.596944    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.597639 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.597152    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.597639 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.597217    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.597639 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.597601    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.597883 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.597647    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-volume\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.597943 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.597888    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.598325 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.598305    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-web-config\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.599045 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.599004    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.599254 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.599222    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-out\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.599983 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.599962    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.603113 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.603093    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sscjm\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-kube-api-access-sscjm\") pod \"alertmanager-main-0\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:46.673513 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:46.673488    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:13:48.814582 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:48.814556    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:13:48.818279 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:48.818247    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7ab118e_f936_441f_af32_bd6dd4a3ceb5.slice/crio-2da03d2f80eb7d855c163a56a6b2e4d611538e2aca609440f5f6582ba354267f WatchSource:0}: Error finding container 2da03d2f80eb7d855c163a56a6b2e4d611538e2aca609440f5f6582ba354267f: Status 404 returned error can't find the container with id 2da03d2f80eb7d855c163a56a6b2e4d611538e2aca609440f5f6582ba354267f
Aug 05 17:13:49.175817 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.175780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-8gjld" event={"ID":"de68f860-081a-403c-8a3d-6299868bc425","Type":"ContainerStarted","Data":"0f44e43d2cb28db89fcf1db8d4b04c3350e61cc46baefbea51ce62eece584e3a"}
Aug 05 17:13:49.177246 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.177218    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hp8ck" event={"ID":"25c6e85a-9ed2-446f-9b1f-fa204867ac7a","Type":"ContainerStarted","Data":"85d0ce8149330813d8227da6517002c960c2a5a0b3785d54dc46cb36924fcf2e"}
Aug 05 17:13:49.177367 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.177272    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:13:49.178584 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.178557    2574 generic.go:356] "Generic (PLEG): container finished" podID="e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb" containerID="95686b2299a464e00e150fc7a78508b82882ed9faa52cd6cd250b574b69ed0b9" exitCode=0
Aug 05 17:13:49.178695 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.178643    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zptlm" event={"ID":"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb","Type":"ContainerDied","Data":"95686b2299a464e00e150fc7a78508b82882ed9faa52cd6cd250b574b69ed0b9"}
Aug 05 17:13:49.179845 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.179800    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerStarted","Data":"2da03d2f80eb7d855c163a56a6b2e4d611538e2aca609440f5f6582ba354267f"}
Aug 05 17:13:49.192480 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.192429    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-8gjld" podStartSLOduration=68.761865052 podStartE2EDuration="1m14.192411792s" totalImagesPullingTime="5.43054674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:49.191752781 +0000 UTC m=+73.891008122" watchObservedRunningTime="2026-08-05 17:13:49.192411792 +0000 UTC m=+73.891667135"
Aug 05 17:13:49.229667 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.229624    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hp8ck" podStartSLOduration=68.812625297 podStartE2EDuration="1m14.229607878s" totalImagesPullingTime="5.416982581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:12:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:49.228548933 +0000 UTC m=+73.927804274" watchObservedRunningTime="2026-08-05 17:13:49.229607878 +0000 UTC m=+73.928863220"
Aug 05 17:13:49.928566 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.928532    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"]
Aug 05 17:13:49.930800 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.930782    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:49.933860 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.933838    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-5vqsv\""
Aug 05 17:13:49.934168 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.934149    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Aug 05 17:13:49.943956 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:49.943936    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"]
Aug 05 17:13:50.019848 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.019825    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thp6d\" (UniqueName: \"kubernetes.io/projected/4e868dc0-89f8-47a4-8ffc-ac39f1b08daf-kube-api-access-thp6d\") pod \"monitoring-plugin-6dffbbd67-8mgrr\" (UID: \"4e868dc0-89f8-47a4-8ffc-ac39f1b08daf\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:50.019934 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.019860    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4e868dc0-89f8-47a4-8ffc-ac39f1b08daf-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-8mgrr\" (UID: \"4e868dc0-89f8-47a4-8ffc-ac39f1b08daf\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:50.120984 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.120961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thp6d\" (UniqueName: \"kubernetes.io/projected/4e868dc0-89f8-47a4-8ffc-ac39f1b08daf-kube-api-access-thp6d\") pod \"monitoring-plugin-6dffbbd67-8mgrr\" (UID: \"4e868dc0-89f8-47a4-8ffc-ac39f1b08daf\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:50.121107 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.121010    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4e868dc0-89f8-47a4-8ffc-ac39f1b08daf-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-8mgrr\" (UID: \"4e868dc0-89f8-47a4-8ffc-ac39f1b08daf\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:50.121151 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:50.121115    2574 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Aug 05 17:13:50.121191 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:13:50.121174    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e868dc0-89f8-47a4-8ffc-ac39f1b08daf-monitoring-plugin-cert podName:4e868dc0-89f8-47a4-8ffc-ac39f1b08daf nodeName:}" failed. No retries permitted until 2026-08-05 17:13:50.621158706 +0000 UTC m=+75.320414026 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/4e868dc0-89f8-47a4-8ffc-ac39f1b08daf-monitoring-plugin-cert") pod "monitoring-plugin-6dffbbd67-8mgrr" (UID: "4e868dc0-89f8-47a4-8ffc-ac39f1b08daf") : secret "monitoring-plugin-cert" not found
Aug 05 17:13:50.135590 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.135573    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thp6d\" (UniqueName: \"kubernetes.io/projected/4e868dc0-89f8-47a4-8ffc-ac39f1b08daf-kube-api-access-thp6d\") pod \"monitoring-plugin-6dffbbd67-8mgrr\" (UID: \"4e868dc0-89f8-47a4-8ffc-ac39f1b08daf\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:50.187926 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.187860    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zptlm" event={"ID":"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb","Type":"ContainerStarted","Data":"8aaacbc717133a35cf16a5bdbf3d3c5f1cee0be66aae7bed2334c61d6d6d5ce4"}
Aug 05 17:13:50.187926 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.187911    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zptlm" event={"ID":"e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb","Type":"ContainerStarted","Data":"e2d9f62b552e9ea93a50a817711a846e6e65f32495148c5ea595c021a56d1376"}
Aug 05 17:13:50.189424 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.189399    2574 generic.go:356] "Generic (PLEG): container finished" podID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerID="ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23" exitCode=0
Aug 05 17:13:50.189518 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.189489    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerDied","Data":"ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23"}
Aug 05 17:13:50.224961 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.224922    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zptlm" podStartSLOduration=2.064472172 podStartE2EDuration="5.224908801s" totalImagesPullingTime="3.091734213s" totalInitContainerRuntime="68.702416ms" isStatefulPod=true podCreationTimestamp="2026-08-05 17:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:50.223995419 +0000 UTC m=+74.923250763" watchObservedRunningTime="2026-08-05 17:13:50.224908801 +0000 UTC m=+74.924164143"
Aug 05 17:13:50.624543 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.624461    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4e868dc0-89f8-47a4-8ffc-ac39f1b08daf-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-8mgrr\" (UID: \"4e868dc0-89f8-47a4-8ffc-ac39f1b08daf\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:50.626713 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.626692    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4e868dc0-89f8-47a4-8ffc-ac39f1b08daf-monitoring-plugin-cert\") pod \"monitoring-plugin-6dffbbd67-8mgrr\" (UID: \"4e868dc0-89f8-47a4-8ffc-ac39f1b08daf\") " pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:50.892081 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:50.891985    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:51.044679 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:51.044505    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"]
Aug 05 17:13:51.047606 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:13:51.047577    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e868dc0_89f8_47a4_8ffc_ac39f1b08daf.slice/crio-49e49329f086df8bf76815cec1d2f47c5ef013c3af8dbaab2d765ca401964782 WatchSource:0}: Error finding container 49e49329f086df8bf76815cec1d2f47c5ef013c3af8dbaab2d765ca401964782: Status 404 returned error can't find the container with id 49e49329f086df8bf76815cec1d2f47c5ef013c3af8dbaab2d765ca401964782
Aug 05 17:13:51.194239 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:51.194204    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr" event={"ID":"4e868dc0-89f8-47a4-8ffc-ac39f1b08daf","Type":"ContainerStarted","Data":"49e49329f086df8bf76815cec1d2f47c5ef013c3af8dbaab2d765ca401964782"}
Aug 05 17:13:52.201761 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:52.201726    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerStarted","Data":"03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b"}
Aug 05 17:13:52.202194 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:52.201794    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerStarted","Data":"12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990"}
Aug 05 17:13:52.202194 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:52.201827    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerStarted","Data":"00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45"}
Aug 05 17:13:52.202194 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:52.201859    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerStarted","Data":"3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9"}
Aug 05 17:13:52.202194 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:52.201886    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerStarted","Data":"cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb"}
Aug 05 17:13:53.208282 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:53.208249    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerStarted","Data":"e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345"}
Aug 05 17:13:53.209748 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:53.209724    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr" event={"ID":"4e868dc0-89f8-47a4-8ffc-ac39f1b08daf","Type":"ContainerStarted","Data":"40ea4806e413484da7f49ff97a46727bf280aa1989448c6c9b8e287ddb156c09"}
Aug 05 17:13:53.209918 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:53.209905    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:53.214602 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:53.214584    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr"
Aug 05 17:13:53.253634 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:53.253595    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.968027256 podStartE2EDuration="7.253582619s" totalImagesPullingTime="3.266009598s" totalInitContainerRuntime="19.545765ms" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:53.25237204 +0000 UTC m=+77.951627382" watchObservedRunningTime="2026-08-05 17:13:53.253582619 +0000 UTC m=+77.952837960"
Aug 05 17:13:53.274079 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:13:53.274040    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-6dffbbd67-8mgrr" podStartSLOduration=2.910391675 podStartE2EDuration="4.274025942s" totalImagesPullingTime="1.363634267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:13:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:13:53.273906239 +0000 UTC m=+77.973161582" watchObservedRunningTime="2026-08-05 17:13:53.274025942 +0000 UTC m=+77.973281276"
Aug 05 17:14:20.192267 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:14:20.192236    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hp8ck"
Aug 05 17:15:05.749519 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:05.749485    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:15:05.750079 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:05.749776    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="alertmanager" containerID="cri-o://cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb" gracePeriod=120
Aug 05 17:15:05.750079 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:05.749838    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy-metric" containerID="cri-o://03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b" gracePeriod=120
Aug 05 17:15:05.750079 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:05.749855    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy-web" containerID="cri-o://00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45" gracePeriod=120
Aug 05 17:15:05.750079 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:05.749887    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy" containerID="cri-o://12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990" gracePeriod=120
Aug 05 17:15:05.750079 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:05.749899    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="config-reloader" containerID="cri-o://3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9" gracePeriod=120
Aug 05 17:15:05.750079 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:05.749953    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="prom-label-proxy" containerID="cri-o://e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345" gracePeriod=120
Aug 05 17:15:06.459403 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:06.459369    2574 generic.go:356] "Generic (PLEG): container finished" podID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerID="e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345" exitCode=0
Aug 05 17:15:06.459403 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:06.459396    2574 generic.go:356] "Generic (PLEG): container finished" podID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerID="12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990" exitCode=0
Aug 05 17:15:06.459403 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:06.459403    2574 generic.go:356] "Generic (PLEG): container finished" podID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerID="3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9" exitCode=0
Aug 05 17:15:06.459403 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:06.459411    2574 generic.go:356] "Generic (PLEG): container finished" podID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerID="cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb" exitCode=0
Aug 05 17:15:06.459658 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:06.459429    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerDied","Data":"e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345"}
Aug 05 17:15:06.459658 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:06.459504    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerDied","Data":"12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990"}
Aug 05 17:15:06.459658 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:06.459530    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerDied","Data":"3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9"}
Aug 05 17:15:06.459658 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:06.459554    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerDied","Data":"cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb"}
Aug 05 17:15:06.977115 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:06.977094    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.044681 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044656    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-tls-assets\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.044681 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044685    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-out\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.044854 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044704    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-main-tls\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.044854 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044729    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.044854 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044771    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-web\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.044854 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044798    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-metrics-client-ca\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.044854 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044816    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-volume\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.044854 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044841    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-cluster-tls-config\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.045170 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044875    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sscjm\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-kube-api-access-sscjm\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.045170 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044898    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-main-db\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.045170 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044928    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-trusted-ca-bundle\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.045170 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.044995    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.045170 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.045058    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-web-config\") pod \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\" (UID: \"d7ab118e-f936-441f-af32-bd6dd4a3ceb5\") "
Aug 05 17:15:07.046379 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.045587    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:15:07.046379 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.046242    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:15:07.046379 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.046340    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:15:07.048255 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.048210    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:07.048404 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.048377    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-out" (OuterVolumeSpecName: "config-out") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:15:07.049002 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.048978    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:15:07.049118 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.049072    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:07.049319 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.049281    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:07.049395 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.049363    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:07.049649 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.049624    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-volume" (OuterVolumeSpecName: "config-volume") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:07.049649 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.049625    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-kube-api-access-sscjm" (OuterVolumeSpecName: "kube-api-access-sscjm") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "kube-api-access-sscjm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:15:07.053705 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.053680    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:07.059119 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.059099    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-web-config" (OuterVolumeSpecName: "web-config") pod "d7ab118e-f936-441f-af32-bd6dd4a3ceb5" (UID: "d7ab118e-f936-441f-af32-bd6dd4a3ceb5"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:15:07.146346 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146281    2574 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-web-config\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146346 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146302    2574 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-tls-assets\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146346 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146312    2574 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-out\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146346 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146320    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-main-tls\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146346 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146330    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146346 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146341    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146346 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146351    2574 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-metrics-client-ca\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146640 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146359    2574 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-config-volume\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146640 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146367    2574 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-cluster-tls-config\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146640 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146375    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sscjm\" (UniqueName: \"kubernetes.io/projected/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-kube-api-access-sscjm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146640 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146383    2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-main-db\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146640 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146392    2574 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.146640 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.146400    2574 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d7ab118e-f936-441f-af32-bd6dd4a3ceb5-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:15:07.466252 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.466218    2574 generic.go:356] "Generic (PLEG): container finished" podID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerID="03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b" exitCode=0
Aug 05 17:15:07.466252 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.466244    2574 generic.go:356] "Generic (PLEG): container finished" podID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerID="00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45" exitCode=0
Aug 05 17:15:07.466431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.466265    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerDied","Data":"03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b"}
Aug 05 17:15:07.466431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.466301    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.466431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.466321    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerDied","Data":"00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45"}
Aug 05 17:15:07.466431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.466349    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d7ab118e-f936-441f-af32-bd6dd4a3ceb5","Type":"ContainerDied","Data":"2da03d2f80eb7d855c163a56a6b2e4d611538e2aca609440f5f6582ba354267f"}
Aug 05 17:15:07.466431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.466379    2574 scope.go:117] "RemoveContainer" containerID="e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345"
Aug 05 17:15:07.473868 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.473848    2574 scope.go:117] "RemoveContainer" containerID="03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b"
Aug 05 17:15:07.480665 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.480652    2574 scope.go:117] "RemoveContainer" containerID="12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990"
Aug 05 17:15:07.486604 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.486588    2574 scope.go:117] "RemoveContainer" containerID="00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45"
Aug 05 17:15:07.492412 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.492393    2574 scope.go:117] "RemoveContainer" containerID="3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9"
Aug 05 17:15:07.495560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.495538    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:15:07.501000 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.500964    2574 scope.go:117] "RemoveContainer" containerID="cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb"
Aug 05 17:15:07.503526 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.503504    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:15:07.508230 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.508209    2574 scope.go:117] "RemoveContainer" containerID="ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23"
Aug 05 17:15:07.514314 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.514298    2574 scope.go:117] "RemoveContainer" containerID="e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345"
Aug 05 17:15:07.514612 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:15:07.514593    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345\": container with ID starting with e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345 not found: ID does not exist" containerID="e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345"
Aug 05 17:15:07.514666 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.514622    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345"} err="failed to get container status \"e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345\": rpc error: code = NotFound desc = could not find container \"e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345\": container with ID starting with e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345 not found: ID does not exist"
Aug 05 17:15:07.514666 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.514655    2574 scope.go:117] "RemoveContainer" containerID="03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b"
Aug 05 17:15:07.514880 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:15:07.514862    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b\": container with ID starting with 03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b not found: ID does not exist" containerID="03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b"
Aug 05 17:15:07.514922 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.514894    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b"} err="failed to get container status \"03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b\": rpc error: code = NotFound desc = could not find container \"03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b\": container with ID starting with 03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b not found: ID does not exist"
Aug 05 17:15:07.514922 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.514911    2574 scope.go:117] "RemoveContainer" containerID="12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990"
Aug 05 17:15:07.515210 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:15:07.515192    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990\": container with ID starting with 12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990 not found: ID does not exist" containerID="12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990"
Aug 05 17:15:07.515279 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.515218    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990"} err="failed to get container status \"12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990\": rpc error: code = NotFound desc = could not find container \"12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990\": container with ID starting with 12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990 not found: ID does not exist"
Aug 05 17:15:07.515279 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.515238    2574 scope.go:117] "RemoveContainer" containerID="00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45"
Aug 05 17:15:07.515488 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:15:07.515472    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45\": container with ID starting with 00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45 not found: ID does not exist" containerID="00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45"
Aug 05 17:15:07.515530 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.515491    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45"} err="failed to get container status \"00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45\": rpc error: code = NotFound desc = could not find container \"00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45\": container with ID starting with 00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45 not found: ID does not exist"
Aug 05 17:15:07.515530 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.515505    2574 scope.go:117] "RemoveContainer" containerID="3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9"
Aug 05 17:15:07.515767 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:15:07.515750    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9\": container with ID starting with 3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9 not found: ID does not exist" containerID="3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9"
Aug 05 17:15:07.515815 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.515771    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9"} err="failed to get container status \"3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9\": rpc error: code = NotFound desc = could not find container \"3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9\": container with ID starting with 3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9 not found: ID does not exist"
Aug 05 17:15:07.515815 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.515786    2574 scope.go:117] "RemoveContainer" containerID="cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb"
Aug 05 17:15:07.515989 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:15:07.515975    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb\": container with ID starting with cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb not found: ID does not exist" containerID="cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb"
Aug 05 17:15:07.516054 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.515993    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb"} err="failed to get container status \"cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb\": rpc error: code = NotFound desc = could not find container \"cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb\": container with ID starting with cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb not found: ID does not exist"
Aug 05 17:15:07.516054 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.516028    2574 scope.go:117] "RemoveContainer" containerID="ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23"
Aug 05 17:15:07.516260 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:15:07.516243    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23\": container with ID starting with ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23 not found: ID does not exist" containerID="ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23"
Aug 05 17:15:07.516301 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.516263    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23"} err="failed to get container status \"ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23\": rpc error: code = NotFound desc = could not find container \"ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23\": container with ID starting with ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23 not found: ID does not exist"
Aug 05 17:15:07.516301 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.516277    2574 scope.go:117] "RemoveContainer" containerID="e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345"
Aug 05 17:15:07.516504 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.516484    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345"} err="failed to get container status \"e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345\": rpc error: code = NotFound desc = could not find container \"e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345\": container with ID starting with e9d8f35d3cc3c756a37fc68e8badeff505a44ad2ae001078a07b817e64de3345 not found: ID does not exist"
Aug 05 17:15:07.516544 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.516505    2574 scope.go:117] "RemoveContainer" containerID="03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b"
Aug 05 17:15:07.516686 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.516670    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b"} err="failed to get container status \"03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b\": rpc error: code = NotFound desc = could not find container \"03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b\": container with ID starting with 03f2fe0360f1663849eca7f82b0fafbee33367573976eb6ca62c5fe6fa114a8b not found: ID does not exist"
Aug 05 17:15:07.516749 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.516693    2574 scope.go:117] "RemoveContainer" containerID="12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990"
Aug 05 17:15:07.516904 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.516886    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990"} err="failed to get container status \"12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990\": rpc error: code = NotFound desc = could not find container \"12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990\": container with ID starting with 12599fc9bf9e2de4262f7f44103af092395292b34e85280606557cfe7ddde990 not found: ID does not exist"
Aug 05 17:15:07.516945 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.516906    2574 scope.go:117] "RemoveContainer" containerID="00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45"
Aug 05 17:15:07.517124 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.517107    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45"} err="failed to get container status \"00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45\": rpc error: code = NotFound desc = could not find container \"00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45\": container with ID starting with 00982d51ab45fc418dd10f72ea2aa78ea15196af2ac296075d533201bd502c45 not found: ID does not exist"
Aug 05 17:15:07.517169 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.517124    2574 scope.go:117] "RemoveContainer" containerID="3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9"
Aug 05 17:15:07.517322 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.517306    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9"} err="failed to get container status \"3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9\": rpc error: code = NotFound desc = could not find container \"3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9\": container with ID starting with 3ad1349acc4f0b758ce7c0083e4f900e2e4744389103ad387986bc19bb8f2dd9 not found: ID does not exist"
Aug 05 17:15:07.517361 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.517322    2574 scope.go:117] "RemoveContainer" containerID="cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb"
Aug 05 17:15:07.517518 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.517497    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb"} err="failed to get container status \"cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb\": rpc error: code = NotFound desc = could not find container \"cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb\": container with ID starting with cf45d49d02998920aa063f7b4c5a2f63bf88c21bfaf7004aa92a6316775c8ccb not found: ID does not exist"
Aug 05 17:15:07.517518 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.517512    2574 scope.go:117] "RemoveContainer" containerID="ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23"
Aug 05 17:15:07.517697 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.517678    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23"} err="failed to get container status \"ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23\": rpc error: code = NotFound desc = could not find container \"ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23\": container with ID starting with ecdd112041af3267f49d4d4282918852624031c9f390ecd7291ddc218e5f8b23 not found: ID does not exist"
Aug 05 17:15:07.557349 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557324    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:15:07.557537 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557527    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy-metric"
Aug 05 17:15:07.557584 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557539    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy-metric"
Aug 05 17:15:07.557584 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557548    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="alertmanager"
Aug 05 17:15:07.557584 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557553    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="alertmanager"
Aug 05 17:15:07.557584 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557561    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy"
Aug 05 17:15:07.557584 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557567    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy"
Aug 05 17:15:07.557584 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557572    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="prom-label-proxy"
Aug 05 17:15:07.557584 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557577    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="prom-label-proxy"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557588    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="init-config-reloader"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557594    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="init-config-reloader"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557600    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="config-reloader"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557605    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="config-reloader"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557611    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy-web"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557616    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy-web"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557654    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy-metric"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557662    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy-web"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557668    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="kube-rbac-proxy"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557673    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="alertmanager"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557679    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="config-reloader"
Aug 05 17:15:07.557785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.557684    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" containerName="prom-label-proxy"
Aug 05 17:15:07.561079 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.561066    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.570298 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.570278    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Aug 05 17:15:07.570402 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.570281    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Aug 05 17:15:07.570613 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.570578    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Aug 05 17:15:07.570658 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.570619    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Aug 05 17:15:07.570862 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.570849    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Aug 05 17:15:07.571095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.571080    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Aug 05 17:15:07.571095 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.571091    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Aug 05 17:15:07.571198 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.571084    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Aug 05 17:15:07.572932 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.572919    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-z7sqb\""
Aug 05 17:15:07.587277 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.587260    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Aug 05 17:15:07.595930 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.595908    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:15:07.650100 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650080    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650210 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650210 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13e36d99-e603-46ab-bcf5-6ff13f01d33d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650210 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13e36d99-e603-46ab-bcf5-6ff13f01d33d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650210 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650166    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-web-config\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650375 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650245    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13e36d99-e603-46ab-bcf5-6ff13f01d33d-config-out\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650375 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650265    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13e36d99-e603-46ab-bcf5-6ff13f01d33d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650375 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650284    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpkrl\" (UniqueName: \"kubernetes.io/projected/13e36d99-e603-46ab-bcf5-6ff13f01d33d-kube-api-access-zpkrl\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650375 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650301    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13e36d99-e603-46ab-bcf5-6ff13f01d33d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650375 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650317    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650375 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650339    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650375 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650360    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-config-volume\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.650582 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.650378    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751097 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-web-config\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751097 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751062    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13e36d99-e603-46ab-bcf5-6ff13f01d33d-config-out\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751097 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13e36d99-e603-46ab-bcf5-6ff13f01d33d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751097 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751093    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpkrl\" (UniqueName: \"kubernetes.io/projected/13e36d99-e603-46ab-bcf5-6ff13f01d33d-kube-api-access-zpkrl\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751110    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13e36d99-e603-46ab-bcf5-6ff13f01d33d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751124    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-config-volume\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751204    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751242    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751276    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751301    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13e36d99-e603-46ab-bcf5-6ff13f01d33d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751329    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13e36d99-e603-46ab-bcf5-6ff13f01d33d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.751812 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.751524    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/13e36d99-e603-46ab-bcf5-6ff13f01d33d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.752663 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.752637    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/13e36d99-e603-46ab-bcf5-6ff13f01d33d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.752844 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.752819    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13e36d99-e603-46ab-bcf5-6ff13f01d33d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.754161 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.754120    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.754161 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.754126    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/13e36d99-e603-46ab-bcf5-6ff13f01d33d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.756037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.754682    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-web-config\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.756037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.754799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.756037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.754844    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/13e36d99-e603-46ab-bcf5-6ff13f01d33d-config-out\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.756037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.754880    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.756037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.754981    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-config-volume\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.756037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.755183    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.756037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.755915    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/13e36d99-e603-46ab-bcf5-6ff13f01d33d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.762892 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.762871    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpkrl\" (UniqueName: \"kubernetes.io/projected/13e36d99-e603-46ab-bcf5-6ff13f01d33d-kube-api-access-zpkrl\") pod \"alertmanager-main-0\" (UID: \"13e36d99-e603-46ab-bcf5-6ff13f01d33d\") " pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:07.853726 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.853693    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7ab118e-f936-441f-af32-bd6dd4a3ceb5" path="/var/lib/kubelet/pods/d7ab118e-f936-441f-af32-bd6dd4a3ceb5/volumes"
Aug 05 17:15:07.869495 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:07.869464    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Aug 05 17:15:08.004760 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:08.004737    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Aug 05 17:15:08.006911 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:15:08.006886    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13e36d99_e603_46ab_bcf5_6ff13f01d33d.slice/crio-9b6cbfb85f4915bab46be5c34be07093046af149f3cb79cba7367e3e2086d8ab WatchSource:0}: Error finding container 9b6cbfb85f4915bab46be5c34be07093046af149f3cb79cba7367e3e2086d8ab: Status 404 returned error can't find the container with id 9b6cbfb85f4915bab46be5c34be07093046af149f3cb79cba7367e3e2086d8ab
Aug 05 17:15:08.472275 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:08.472241    2574 generic.go:356] "Generic (PLEG): container finished" podID="13e36d99-e603-46ab-bcf5-6ff13f01d33d" containerID="fe1bc7c8f6fd90371ddd6e9d6da8616574fef817d2f72c2995d0b9ce6a304027" exitCode=0
Aug 05 17:15:08.472427 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:08.472307    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e36d99-e603-46ab-bcf5-6ff13f01d33d","Type":"ContainerDied","Data":"fe1bc7c8f6fd90371ddd6e9d6da8616574fef817d2f72c2995d0b9ce6a304027"}
Aug 05 17:15:08.472427 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:08.472353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e36d99-e603-46ab-bcf5-6ff13f01d33d","Type":"ContainerStarted","Data":"9b6cbfb85f4915bab46be5c34be07093046af149f3cb79cba7367e3e2086d8ab"}
Aug 05 17:15:09.478556 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.478525    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e36d99-e603-46ab-bcf5-6ff13f01d33d","Type":"ContainerStarted","Data":"ba69eec398b1cec2518dd31a3511d80366d80b81a98c8c2b6cb3ad190f36f030"}
Aug 05 17:15:09.478926 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.478583    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e36d99-e603-46ab-bcf5-6ff13f01d33d","Type":"ContainerStarted","Data":"d3a7fe3a753cd3d6e70a11e281ff0f0c53a0a32512e04f0b8194f595c7745cc0"}
Aug 05 17:15:09.478926 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.478609    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e36d99-e603-46ab-bcf5-6ff13f01d33d","Type":"ContainerStarted","Data":"f52272be5a709ffbe180fca2da00c4aa0181b9b0869986242c8ca54f7760d987"}
Aug 05 17:15:09.478926 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.478634    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e36d99-e603-46ab-bcf5-6ff13f01d33d","Type":"ContainerStarted","Data":"fdc71f639febc7fcb9eab98ee082e6854e69e1fac415c6ff1010956624ba7fea"}
Aug 05 17:15:09.478926 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.478660    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e36d99-e603-46ab-bcf5-6ff13f01d33d","Type":"ContainerStarted","Data":"5ba2d1b1874e552cdc5bf0a18a4f488eedba9aaf3328a799248559b417bfe7d8"}
Aug 05 17:15:09.478926 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.478683    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"13e36d99-e603-46ab-bcf5-6ff13f01d33d","Type":"ContainerStarted","Data":"42c0c6e41094a084d6cfc228a12ced509189a6ad5e9712aa428a2582e2a337c3"}
Aug 05 17:15:09.522964 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.522917    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.506507998 podStartE2EDuration="2.52290488s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.396882ms" isStatefulPod=false podCreationTimestamp="2026-08-05 17:15:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:15:09.520538374 +0000 UTC m=+154.219793717" watchObservedRunningTime="2026-08-05 17:15:09.52290488 +0000 UTC m=+154.222160221"
Aug 05 17:15:09.746753 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.746669    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5b98b597b4-dls9w"]
Aug 05 17:15:09.748911 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.748896    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.754972 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.754951    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Aug 05 17:15:09.755185 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.755172    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Aug 05 17:15:09.755719 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.755686    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4g22p\""
Aug 05 17:15:09.756107 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.755828    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Aug 05 17:15:09.756177 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.756133    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Aug 05 17:15:09.756384 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.756366    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Aug 05 17:15:09.766085 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.766059    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-telemeter-client-tls\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.766187 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.766121    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.766187 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.766170    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e485294-1501-4a6d-96ca-809e78b097f5-metrics-client-ca\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.766289 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.766190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-federate-client-tls\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.766289 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.766207    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tlpk\" (UniqueName: \"kubernetes.io/projected/2e485294-1501-4a6d-96ca-809e78b097f5-kube-api-access-9tlpk\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.766289 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.766235    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e485294-1501-4a6d-96ca-809e78b097f5-serving-certs-ca-bundle\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.766289 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.766253    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-secret-telemeter-client\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.766437 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.766287    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e485294-1501-4a6d-96ca-809e78b097f5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.767302 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.767286    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Aug 05 17:15:09.778612 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.778589    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b98b597b4-dls9w"]
Aug 05 17:15:09.866997 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.866965    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e485294-1501-4a6d-96ca-809e78b097f5-metrics-client-ca\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.866997 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.866999    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-federate-client-tls\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.867200 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.867038    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tlpk\" (UniqueName: \"kubernetes.io/projected/2e485294-1501-4a6d-96ca-809e78b097f5-kube-api-access-9tlpk\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.867200 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.867102    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e485294-1501-4a6d-96ca-809e78b097f5-serving-certs-ca-bundle\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.867200 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.867117    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-secret-telemeter-client\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.867373 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.867205    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e485294-1501-4a6d-96ca-809e78b097f5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.867373 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.867310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-telemeter-client-tls\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.867467 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.867393    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.867811 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.867787    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2e485294-1501-4a6d-96ca-809e78b097f5-metrics-client-ca\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.867976 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.867787    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e485294-1501-4a6d-96ca-809e78b097f5-serving-certs-ca-bundle\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.868494 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.868471    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2e485294-1501-4a6d-96ca-809e78b097f5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.869740 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.869716    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-federate-client-tls\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.869856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.869842    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-secret-telemeter-client\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.870149 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.870127    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.870187 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.870136    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2e485294-1501-4a6d-96ca-809e78b097f5-telemeter-client-tls\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:09.881856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:09.881835    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tlpk\" (UniqueName: \"kubernetes.io/projected/2e485294-1501-4a6d-96ca-809e78b097f5-kube-api-access-9tlpk\") pod \"telemeter-client-5b98b597b4-dls9w\" (UID: \"2e485294-1501-4a6d-96ca-809e78b097f5\") " pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:10.063940 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:10.063873    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w"
Aug 05 17:15:10.208919 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:10.208886    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5b98b597b4-dls9w"]
Aug 05 17:15:10.212897 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:15:10.212874    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e485294_1501_4a6d_96ca_809e78b097f5.slice/crio-f8b413eb2fe2b954b6d4ba839fe2005ec07f4ff0c679061a36dfe86f6a53dbfe WatchSource:0}: Error finding container f8b413eb2fe2b954b6d4ba839fe2005ec07f4ff0c679061a36dfe86f6a53dbfe: Status 404 returned error can't find the container with id f8b413eb2fe2b954b6d4ba839fe2005ec07f4ff0c679061a36dfe86f6a53dbfe
Aug 05 17:15:10.483444 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:10.483409    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w" event={"ID":"2e485294-1501-4a6d-96ca-809e78b097f5","Type":"ContainerStarted","Data":"f8b413eb2fe2b954b6d4ba839fe2005ec07f4ff0c679061a36dfe86f6a53dbfe"}
Aug 05 17:15:12.493834 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:12.493802    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w" event={"ID":"2e485294-1501-4a6d-96ca-809e78b097f5","Type":"ContainerStarted","Data":"bd58a73a89f55922499a2b67c1c1d3002e83e3d103f6882d02c01cc79061c089"}
Aug 05 17:15:12.494245 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:12.493852    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w" event={"ID":"2e485294-1501-4a6d-96ca-809e78b097f5","Type":"ContainerStarted","Data":"775e13c11a8f0b155c963d1d3e1f0e1e055c86e483272597b0be1f8116ce25d4"}
Aug 05 17:15:12.494245 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:12.493895    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w" event={"ID":"2e485294-1501-4a6d-96ca-809e78b097f5","Type":"ContainerStarted","Data":"0b763fd4494f3ed0bcbb36a13a84475f9484b955dbad9de08ea739457aa081d3"}
Aug 05 17:15:12.527348 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:15:12.527302    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5b98b597b4-dls9w" podStartSLOduration=1.778959548 podStartE2EDuration="3.527286043s" totalImagesPullingTime="1.748326495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:15:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:15:12.526167498 +0000 UTC m=+157.225422840" watchObservedRunningTime="2026-08-05 17:15:12.527286043 +0000 UTC m=+157.226541384"
Aug 05 17:17:35.664952 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:17:35.664923    2574 kubelet.go:1648] "Image garbage collection succeeded"
Aug 05 17:20:37.614589 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.614554    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-trkm8"]
Aug 05 17:20:37.616575 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.616556    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:37.619593 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.619571    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-m2mdg\""
Aug 05 17:20:37.619593 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.619580    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Aug 05 17:20:37.620277 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.620262    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Aug 05 17:20:37.632708 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.632685    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-trkm8"]
Aug 05 17:20:37.657766 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.657741    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2148ec0d-0e45-44be-9842-f429b83e04aa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-trkm8\" (UID: \"2148ec0d-0e45-44be-9842-f429b83e04aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:37.657881 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.657794    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drsdj\" (UniqueName: \"kubernetes.io/projected/2148ec0d-0e45-44be-9842-f429b83e04aa-kube-api-access-drsdj\") pod \"cert-manager-webhook-6486ffd997-trkm8\" (UID: \"2148ec0d-0e45-44be-9842-f429b83e04aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:37.758631 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.758602    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drsdj\" (UniqueName: \"kubernetes.io/projected/2148ec0d-0e45-44be-9842-f429b83e04aa-kube-api-access-drsdj\") pod \"cert-manager-webhook-6486ffd997-trkm8\" (UID: \"2148ec0d-0e45-44be-9842-f429b83e04aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:37.758785 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.758654    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2148ec0d-0e45-44be-9842-f429b83e04aa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-trkm8\" (UID: \"2148ec0d-0e45-44be-9842-f429b83e04aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:37.775485 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.775455    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2148ec0d-0e45-44be-9842-f429b83e04aa-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-trkm8\" (UID: \"2148ec0d-0e45-44be-9842-f429b83e04aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:37.775678 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.775657    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drsdj\" (UniqueName: \"kubernetes.io/projected/2148ec0d-0e45-44be-9842-f429b83e04aa-kube-api-access-drsdj\") pod \"cert-manager-webhook-6486ffd997-trkm8\" (UID: \"2148ec0d-0e45-44be-9842-f429b83e04aa\") " pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:37.925028 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:37.924986    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:38.047261 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:38.047228    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-trkm8"]
Aug 05 17:20:38.050116 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:20:38.050090    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2148ec0d_0e45_44be_9842_f429b83e04aa.slice/crio-1fff042367d9724245603ae0c0492e0e55dda733db1002d09991262db5eda4e5 WatchSource:0}: Error finding container 1fff042367d9724245603ae0c0492e0e55dda733db1002d09991262db5eda4e5: Status 404 returned error can't find the container with id 1fff042367d9724245603ae0c0492e0e55dda733db1002d09991262db5eda4e5
Aug 05 17:20:38.052539 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:38.052520    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:20:38.575996 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:38.575958    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8" event={"ID":"2148ec0d-0e45-44be-9842-f429b83e04aa","Type":"ContainerStarted","Data":"1fff042367d9724245603ae0c0492e0e55dda733db1002d09991262db5eda4e5"}
Aug 05 17:20:42.592743 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:42.592654    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8" event={"ID":"2148ec0d-0e45-44be-9842-f429b83e04aa","Type":"ContainerStarted","Data":"d4b978499e53435e6fb65d7faf066f9f082b8a43a041af570c7ab452425c751c"}
Aug 05 17:20:42.593124 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:42.592769    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:42.613540 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:42.613495    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8" podStartSLOduration=1.48847375 podStartE2EDuration="5.613483097s" totalImagesPullingTime="4.125009347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:42.613234217 +0000 UTC m=+487.312489570" watchObservedRunningTime="2026-08-05 17:20:42.613483097 +0000 UTC m=+487.312738439"
Aug 05 17:20:48.378549 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.378506    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hp2nb"]
Aug 05 17:20:48.380767 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.380752    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hp2nb"
Aug 05 17:20:48.383188 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.383168    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-w2dt7\""
Aug 05 17:20:48.391186 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.391165    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hp2nb"]
Aug 05 17:20:48.441286 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.441256    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w67sh\" (UniqueName: \"kubernetes.io/projected/6f393882-e396-45ea-9d90-12d404f04963-kube-api-access-w67sh\") pod \"cert-manager-55b4d954d5-hp2nb\" (UID: \"6f393882-e396-45ea-9d90-12d404f04963\") " pod="cert-manager/cert-manager-55b4d954d5-hp2nb"
Aug 05 17:20:48.441402 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.441315    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f393882-e396-45ea-9d90-12d404f04963-bound-sa-token\") pod \"cert-manager-55b4d954d5-hp2nb\" (UID: \"6f393882-e396-45ea-9d90-12d404f04963\") " pod="cert-manager/cert-manager-55b4d954d5-hp2nb"
Aug 05 17:20:48.542572 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.542538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w67sh\" (UniqueName: \"kubernetes.io/projected/6f393882-e396-45ea-9d90-12d404f04963-kube-api-access-w67sh\") pod \"cert-manager-55b4d954d5-hp2nb\" (UID: \"6f393882-e396-45ea-9d90-12d404f04963\") " pod="cert-manager/cert-manager-55b4d954d5-hp2nb"
Aug 05 17:20:48.542724 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.542592    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f393882-e396-45ea-9d90-12d404f04963-bound-sa-token\") pod \"cert-manager-55b4d954d5-hp2nb\" (UID: \"6f393882-e396-45ea-9d90-12d404f04963\") " pod="cert-manager/cert-manager-55b4d954d5-hp2nb"
Aug 05 17:20:48.550833 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.550805    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6f393882-e396-45ea-9d90-12d404f04963-bound-sa-token\") pod \"cert-manager-55b4d954d5-hp2nb\" (UID: \"6f393882-e396-45ea-9d90-12d404f04963\") " pod="cert-manager/cert-manager-55b4d954d5-hp2nb"
Aug 05 17:20:48.550939 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.550887    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w67sh\" (UniqueName: \"kubernetes.io/projected/6f393882-e396-45ea-9d90-12d404f04963-kube-api-access-w67sh\") pod \"cert-manager-55b4d954d5-hp2nb\" (UID: \"6f393882-e396-45ea-9d90-12d404f04963\") " pod="cert-manager/cert-manager-55b4d954d5-hp2nb"
Aug 05 17:20:48.597943 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.597926    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-trkm8"
Aug 05 17:20:48.689425 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.689400    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-hp2nb"
Aug 05 17:20:48.809278 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:48.809234    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-hp2nb"]
Aug 05 17:20:48.812085 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:20:48.812059    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f393882_e396_45ea_9d90_12d404f04963.slice/crio-134eb37122656ae5fc599eda8280f11990ba69619f8ad981ef09e681f4dccc08 WatchSource:0}: Error finding container 134eb37122656ae5fc599eda8280f11990ba69619f8ad981ef09e681f4dccc08: Status 404 returned error can't find the container with id 134eb37122656ae5fc599eda8280f11990ba69619f8ad981ef09e681f4dccc08
Aug 05 17:20:49.618294 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:49.618261    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hp2nb" event={"ID":"6f393882-e396-45ea-9d90-12d404f04963","Type":"ContainerStarted","Data":"f6cd718291d8bf8283d31310107707769b8fe9810ade068d7fa78653454bc5d1"}
Aug 05 17:20:49.618294 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:49.618298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-hp2nb" event={"ID":"6f393882-e396-45ea-9d90-12d404f04963","Type":"ContainerStarted","Data":"134eb37122656ae5fc599eda8280f11990ba69619f8ad981ef09e681f4dccc08"}
Aug 05 17:20:49.633844 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:20:49.633789    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-hp2nb" podStartSLOduration=1.633770831 podStartE2EDuration="1.633770831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:20:49.633277608 +0000 UTC m=+494.332532931" watchObservedRunningTime="2026-08-05 17:20:49.633770831 +0000 UTC m=+494.333026175"
Aug 05 17:21:14.260586 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.260552    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"]
Aug 05 17:21:14.262902 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.262885    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.265379 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.265352    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-7pxwt\""
Aug 05 17:21:14.265379 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.265352    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Aug 05 17:21:14.271538 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.271515    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Aug 05 17:21:14.271653 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.271595    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Aug 05 17:21:14.271729 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.271689    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Aug 05 17:21:14.271788 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.271752    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Aug 05 17:21:14.283646 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.283625    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"]
Aug 05 17:21:14.427106 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.427069    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88467a88-d0f1-4fb6-816c-5b8d2aacf890-metrics-cert\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.427264 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.427137    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88467a88-d0f1-4fb6-816c-5b8d2aacf890-cert\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.427264 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.427204    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5t5j\" (UniqueName: \"kubernetes.io/projected/88467a88-d0f1-4fb6-816c-5b8d2aacf890-kube-api-access-l5t5j\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.427264 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.427254    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88467a88-d0f1-4fb6-816c-5b8d2aacf890-manager-config\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.528405 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.528332    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88467a88-d0f1-4fb6-816c-5b8d2aacf890-metrics-cert\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.528405 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.528382    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88467a88-d0f1-4fb6-816c-5b8d2aacf890-cert\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.528405 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.528399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5t5j\" (UniqueName: \"kubernetes.io/projected/88467a88-d0f1-4fb6-816c-5b8d2aacf890-kube-api-access-l5t5j\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.528646 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.528423    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88467a88-d0f1-4fb6-816c-5b8d2aacf890-manager-config\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.529127 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.529108    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/88467a88-d0f1-4fb6-816c-5b8d2aacf890-manager-config\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.530894 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.530862    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/88467a88-d0f1-4fb6-816c-5b8d2aacf890-metrics-cert\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.530998 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.530910    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/88467a88-d0f1-4fb6-816c-5b8d2aacf890-cert\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.611778 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.611746    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5t5j\" (UniqueName: \"kubernetes.io/projected/88467a88-d0f1-4fb6-816c-5b8d2aacf890-kube-api-access-l5t5j\") pod \"lws-controller-manager-77c595c7bb-7cb8n\" (UID: \"88467a88-d0f1-4fb6-816c-5b8d2aacf890\") " pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:14.877210 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:14.877131    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:15.000530 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:15.000491    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"]
Aug 05 17:21:15.003173 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:21:15.003148    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88467a88_d0f1_4fb6_816c_5b8d2aacf890.slice/crio-1812be4eb2732f88ec01ee9dc37c78da4f1411d42fd4fbda3f54290bd214d935 WatchSource:0}: Error finding container 1812be4eb2732f88ec01ee9dc37c78da4f1411d42fd4fbda3f54290bd214d935: Status 404 returned error can't find the container with id 1812be4eb2732f88ec01ee9dc37c78da4f1411d42fd4fbda3f54290bd214d935
Aug 05 17:21:15.711709 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:15.711672    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n" event={"ID":"88467a88-d0f1-4fb6-816c-5b8d2aacf890","Type":"ContainerStarted","Data":"1812be4eb2732f88ec01ee9dc37c78da4f1411d42fd4fbda3f54290bd214d935"}
Aug 05 17:21:18.723370 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:18.723333    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n" event={"ID":"88467a88-d0f1-4fb6-816c-5b8d2aacf890","Type":"ContainerStarted","Data":"a3fdd5650b4923159ddbe0fb3385b7e3a1b2072f60caf718b4c967aa25d92b5c"}
Aug 05 17:21:18.723770 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:18.723433    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:18.742734 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:18.742691    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n" podStartSLOduration=1.450339121 podStartE2EDuration="4.742677512s" totalImagesPullingTime="3.292338391s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:21:18.741586786 +0000 UTC m=+523.440842127" watchObservedRunningTime="2026-08-05 17:21:18.742677512 +0000 UTC m=+523.441932853"
Aug 05 17:21:29.730057 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:29.729969    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-77c595c7bb-7cb8n"
Aug 05 17:21:55.759058 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.759006    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"]
Aug 05 17:21:55.767113 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.767087    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:55.772383 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.772357    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-96dlj\""
Aug 05 17:21:55.773194 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.773171    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Aug 05 17:21:55.773316 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.773223    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Aug 05 17:21:55.773316 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.773302    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Aug 05 17:21:55.773427 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.773318    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Aug 05 17:21:55.776536 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.776515    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"]
Aug 05 17:21:55.830846 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.830817    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2v9h\" (UniqueName: \"kubernetes.io/projected/a81ccf7e-3e7f-476e-8615-2e512c116572-kube-api-access-w2v9h\") pod \"kuadrant-console-plugin-598f86468b-nphpf\" (UID: \"a81ccf7e-3e7f-476e-8615-2e512c116572\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:55.830987 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.830888    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a81ccf7e-3e7f-476e-8615-2e512c116572-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-nphpf\" (UID: \"a81ccf7e-3e7f-476e-8615-2e512c116572\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:55.830987 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.830947    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a81ccf7e-3e7f-476e-8615-2e512c116572-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-nphpf\" (UID: \"a81ccf7e-3e7f-476e-8615-2e512c116572\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:55.931842 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.931818    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a81ccf7e-3e7f-476e-8615-2e512c116572-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-nphpf\" (UID: \"a81ccf7e-3e7f-476e-8615-2e512c116572\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:55.931980 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.931852    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2v9h\" (UniqueName: \"kubernetes.io/projected/a81ccf7e-3e7f-476e-8615-2e512c116572-kube-api-access-w2v9h\") pod \"kuadrant-console-plugin-598f86468b-nphpf\" (UID: \"a81ccf7e-3e7f-476e-8615-2e512c116572\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:55.931980 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.931890    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a81ccf7e-3e7f-476e-8615-2e512c116572-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-nphpf\" (UID: \"a81ccf7e-3e7f-476e-8615-2e512c116572\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:55.932493 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.932467    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a81ccf7e-3e7f-476e-8615-2e512c116572-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-nphpf\" (UID: \"a81ccf7e-3e7f-476e-8615-2e512c116572\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:55.934248 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.934228    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a81ccf7e-3e7f-476e-8615-2e512c116572-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-nphpf\" (UID: \"a81ccf7e-3e7f-476e-8615-2e512c116572\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:55.944140 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:55.944117    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2v9h\" (UniqueName: \"kubernetes.io/projected/a81ccf7e-3e7f-476e-8615-2e512c116572-kube-api-access-w2v9h\") pod \"kuadrant-console-plugin-598f86468b-nphpf\" (UID: \"a81ccf7e-3e7f-476e-8615-2e512c116572\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:56.082259 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:56.082174    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"
Aug 05 17:21:56.212124 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:56.212096    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf"]
Aug 05 17:21:56.215033 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:21:56.214993    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda81ccf7e_3e7f_476e_8615_2e512c116572.slice/crio-2a9b9157133fdcf6dfcbaab90d6605eeb79f497df4030fab845acc280eda71cf WatchSource:0}: Error finding container 2a9b9157133fdcf6dfcbaab90d6605eeb79f497df4030fab845acc280eda71cf: Status 404 returned error can't find the container with id 2a9b9157133fdcf6dfcbaab90d6605eeb79f497df4030fab845acc280eda71cf
Aug 05 17:21:56.858072 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:21:56.858010    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf" event={"ID":"a81ccf7e-3e7f-476e-8615-2e512c116572","Type":"ContainerStarted","Data":"2a9b9157133fdcf6dfcbaab90d6605eeb79f497df4030fab845acc280eda71cf"}
Aug 05 17:22:04.889364 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:04.889326    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf" event={"ID":"a81ccf7e-3e7f-476e-8615-2e512c116572","Type":"ContainerStarted","Data":"b53a9c4874c9b7546ff74fc2768059840a98524c5dec62c8c0527b607536296d"}
Aug 05 17:22:04.912494 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:04.912444    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-nphpf" podStartSLOduration=2.070902007 podStartE2EDuration="9.912425346s" totalImagesPullingTime="7.841523339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:21:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:04.911648424 +0000 UTC m=+569.610903766" watchObservedRunningTime="2026-08-05 17:22:04.912425346 +0000 UTC m=+569.611680690"
Aug 05 17:22:35.510580 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.510544    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6w884"]
Aug 05 17:22:35.518677 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.518657    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:35.520979 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.520957    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 05 17:22:35.526876 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.526852    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6w884"]
Aug 05 17:22:35.602678 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.602644    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6w884"]
Aug 05 17:22:35.631867 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.631841    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk2n6\" (UniqueName: \"kubernetes.io/projected/b39787b2-4b15-4581-9544-cfe9044880f4-kube-api-access-hk2n6\") pod \"limitador-limitador-859c4d96d6-6w884\" (UID: \"b39787b2-4b15-4581-9544-cfe9044880f4\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:35.631984 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.631904    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b39787b2-4b15-4581-9544-cfe9044880f4-config-file\") pod \"limitador-limitador-859c4d96d6-6w884\" (UID: \"b39787b2-4b15-4581-9544-cfe9044880f4\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:35.732719 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.732695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk2n6\" (UniqueName: \"kubernetes.io/projected/b39787b2-4b15-4581-9544-cfe9044880f4-kube-api-access-hk2n6\") pod \"limitador-limitador-859c4d96d6-6w884\" (UID: \"b39787b2-4b15-4581-9544-cfe9044880f4\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:35.732883 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.732754    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b39787b2-4b15-4581-9544-cfe9044880f4-config-file\") pod \"limitador-limitador-859c4d96d6-6w884\" (UID: \"b39787b2-4b15-4581-9544-cfe9044880f4\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:35.734778 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.734762    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Aug 05 17:22:35.740405 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.740387    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk2n6\" (UniqueName: \"kubernetes.io/projected/b39787b2-4b15-4581-9544-cfe9044880f4-kube-api-access-hk2n6\") pod \"limitador-limitador-859c4d96d6-6w884\" (UID: \"b39787b2-4b15-4581-9544-cfe9044880f4\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:35.743571 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.743553    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b39787b2-4b15-4581-9544-cfe9044880f4-config-file\") pod \"limitador-limitador-859c4d96d6-6w884\" (UID: \"b39787b2-4b15-4581-9544-cfe9044880f4\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:35.830284 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.830201    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:35.956047 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:35.956006    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6w884"]
Aug 05 17:22:35.958358 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:22:35.958331    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb39787b2_4b15_4581_9544_cfe9044880f4.slice/crio-c2d77339b113d3c93708b340c216b2f37550dc736719761bf7815d1515864052 WatchSource:0}: Error finding container c2d77339b113d3c93708b340c216b2f37550dc736719761bf7815d1515864052: Status 404 returned error can't find the container with id c2d77339b113d3c93708b340c216b2f37550dc736719761bf7815d1515864052
Aug 05 17:22:36.001928 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.001901    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884" event={"ID":"b39787b2-4b15-4581-9544-cfe9044880f4","Type":"ContainerStarted","Data":"c2d77339b113d3c93708b340c216b2f37550dc736719761bf7815d1515864052"}
Aug 05 17:22:36.514225 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.514194    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cskjl"]
Aug 05 17:22:36.518477 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.518461    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-cskjl"
Aug 05 17:22:36.520597 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.520576    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-h6p9s\""
Aug 05 17:22:36.525616 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.525595    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cskjl"]
Aug 05 17:22:36.640066 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.640038    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwfs6\" (UniqueName: \"kubernetes.io/projected/d78b0a39-e5fa-4c06-8f20-77c85181d6e2-kube-api-access-jwfs6\") pod \"authorino-c548b5d7d-cskjl\" (UID: \"d78b0a39-e5fa-4c06-8f20-77c85181d6e2\") " pod="kuadrant-system/authorino-c548b5d7d-cskjl"
Aug 05 17:22:36.741448 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.741418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwfs6\" (UniqueName: \"kubernetes.io/projected/d78b0a39-e5fa-4c06-8f20-77c85181d6e2-kube-api-access-jwfs6\") pod \"authorino-c548b5d7d-cskjl\" (UID: \"d78b0a39-e5fa-4c06-8f20-77c85181d6e2\") " pod="kuadrant-system/authorino-c548b5d7d-cskjl"
Aug 05 17:22:36.751158 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.751133    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwfs6\" (UniqueName: \"kubernetes.io/projected/d78b0a39-e5fa-4c06-8f20-77c85181d6e2-kube-api-access-jwfs6\") pod \"authorino-c548b5d7d-cskjl\" (UID: \"d78b0a39-e5fa-4c06-8f20-77c85181d6e2\") " pod="kuadrant-system/authorino-c548b5d7d-cskjl"
Aug 05 17:22:36.828229 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.827956    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-cskjl"
Aug 05 17:22:36.981218 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:36.981184    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cskjl"]
Aug 05 17:22:36.985438 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:22:36.985405    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd78b0a39_e5fa_4c06_8f20_77c85181d6e2.slice/crio-8b867569e2b3afbd77017e82c559ae96c20cb7eecf05bfb8ee5545b64b60fe33 WatchSource:0}: Error finding container 8b867569e2b3afbd77017e82c559ae96c20cb7eecf05bfb8ee5545b64b60fe33: Status 404 returned error can't find the container with id 8b867569e2b3afbd77017e82c559ae96c20cb7eecf05bfb8ee5545b64b60fe33
Aug 05 17:22:37.007944 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:37.007911    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-cskjl" event={"ID":"d78b0a39-e5fa-4c06-8f20-77c85181d6e2","Type":"ContainerStarted","Data":"8b867569e2b3afbd77017e82c559ae96c20cb7eecf05bfb8ee5545b64b60fe33"}
Aug 05 17:22:39.019231 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:39.019187    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884" event={"ID":"b39787b2-4b15-4581-9544-cfe9044880f4","Type":"ContainerStarted","Data":"e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1"}
Aug 05 17:22:39.019686 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:39.019255    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:39.038273 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:39.038213    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884" podStartSLOduration=1.508251368 podStartE2EDuration="4.038194269s" totalImagesPullingTime="2.529942901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:39.036417955 +0000 UTC m=+603.735673298" watchObservedRunningTime="2026-08-05 17:22:39.038194269 +0000 UTC m=+603.737449614"
Aug 05 17:22:40.025302 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:40.025258    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-cskjl" event={"ID":"d78b0a39-e5fa-4c06-8f20-77c85181d6e2","Type":"ContainerStarted","Data":"ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc"}
Aug 05 17:22:40.045149 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:40.045104    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c548b5d7d-cskjl" podStartSLOduration=1.4853587 podStartE2EDuration="4.045088398s" totalImagesPullingTime="2.559729698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:22:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:22:40.042952804 +0000 UTC m=+604.742208146" watchObservedRunningTime="2026-08-05 17:22:40.045088398 +0000 UTC m=+604.744343739"
Aug 05 17:22:49.334408 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:49.334373    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6w884"]
Aug 05 17:22:49.334881 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:49.334597    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884" podUID="b39787b2-4b15-4581-9544-cfe9044880f4" containerName="limitador" containerID="cri-o://e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1" gracePeriod=30
Aug 05 17:22:49.335311 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:49.335282    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:49.881889 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:49.881864    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:49.945732 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:49.945704    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b39787b2-4b15-4581-9544-cfe9044880f4-config-file\") pod \"b39787b2-4b15-4581-9544-cfe9044880f4\" (UID: \"b39787b2-4b15-4581-9544-cfe9044880f4\") "
Aug 05 17:22:49.945910 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:49.945756    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk2n6\" (UniqueName: \"kubernetes.io/projected/b39787b2-4b15-4581-9544-cfe9044880f4-kube-api-access-hk2n6\") pod \"b39787b2-4b15-4581-9544-cfe9044880f4\" (UID: \"b39787b2-4b15-4581-9544-cfe9044880f4\") "
Aug 05 17:22:49.949103 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:49.946481    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b39787b2-4b15-4581-9544-cfe9044880f4-config-file" (OuterVolumeSpecName: "config-file") pod "b39787b2-4b15-4581-9544-cfe9044880f4" (UID: "b39787b2-4b15-4581-9544-cfe9044880f4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:22:49.949103 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:49.948822    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b39787b2-4b15-4581-9544-cfe9044880f4-kube-api-access-hk2n6" (OuterVolumeSpecName: "kube-api-access-hk2n6") pod "b39787b2-4b15-4581-9544-cfe9044880f4" (UID: "b39787b2-4b15-4581-9544-cfe9044880f4"). InnerVolumeSpecName "kube-api-access-hk2n6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:22:50.047321 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.047276    2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b39787b2-4b15-4581-9544-cfe9044880f4-config-file\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:22:50.047472 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.047339    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk2n6\" (UniqueName: \"kubernetes.io/projected/b39787b2-4b15-4581-9544-cfe9044880f4-kube-api-access-hk2n6\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:22:50.060495 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.060463    2574 generic.go:356] "Generic (PLEG): container finished" podID="b39787b2-4b15-4581-9544-cfe9044880f4" containerID="e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1" exitCode=0
Aug 05 17:22:50.060610 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.060524    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884"
Aug 05 17:22:50.060610 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.060536    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884" event={"ID":"b39787b2-4b15-4581-9544-cfe9044880f4","Type":"ContainerDied","Data":"e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1"}
Aug 05 17:22:50.060610 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.060602    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-6w884" event={"ID":"b39787b2-4b15-4581-9544-cfe9044880f4","Type":"ContainerDied","Data":"c2d77339b113d3c93708b340c216b2f37550dc736719761bf7815d1515864052"}
Aug 05 17:22:50.060706 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.060623    2574 scope.go:117] "RemoveContainer" containerID="e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1"
Aug 05 17:22:50.069515 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.069491    2574 scope.go:117] "RemoveContainer" containerID="e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1"
Aug 05 17:22:50.069782 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:22:50.069763    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1\": container with ID starting with e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1 not found: ID does not exist" containerID="e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1"
Aug 05 17:22:50.069836 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.069790    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1"} err="failed to get container status \"e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1\": rpc error: code = NotFound desc = could not find container \"e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1\": container with ID starting with e671c7c6b47db69abd1c7221bedc5ea72cfb86457b7ae53ec50c7fe79d0257b1 not found: ID does not exist"
Aug 05 17:22:50.080933 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.080910    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6w884"]
Aug 05 17:22:50.086611 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:50.086592    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-6w884"]
Aug 05 17:22:51.854696 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:22:51.854659    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b39787b2-4b15-4581-9544-cfe9044880f4" path="/var/lib/kubelet/pods/b39787b2-4b15-4581-9544-cfe9044880f4/volumes"
Aug 05 17:23:19.665009 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.664972    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"]
Aug 05 17:23:19.665560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.665411    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b39787b2-4b15-4581-9544-cfe9044880f4" containerName="limitador"
Aug 05 17:23:19.665560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.665429    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b39787b2-4b15-4581-9544-cfe9044880f4" containerName="limitador"
Aug 05 17:23:19.665560 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.665504    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b39787b2-4b15-4581-9544-cfe9044880f4" containerName="limitador"
Aug 05 17:23:19.667598 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.667575    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.671126 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.671095    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Aug 05 17:23:19.671246 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.671208    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Aug 05 17:23:19.671315 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.671276    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-777hc\""
Aug 05 17:23:19.671406 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.671389    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Aug 05 17:23:19.671698 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.671675    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Aug 05 17:23:19.671799 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.671725    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Aug 05 17:23:19.671997 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.671983    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Aug 05 17:23:19.682914 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.682888    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"]
Aug 05 17:23:19.775049 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.775001    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/edae28b3-e05b-4992-91a1-2cc8c9645f36-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.775049 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.775054    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.775261 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.775091    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.775261 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.775147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/edae28b3-e05b-4992-91a1-2cc8c9645f36-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.775261 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.775202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qscxh\" (UniqueName: \"kubernetes.io/projected/edae28b3-e05b-4992-91a1-2cc8c9645f36-kube-api-access-qscxh\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.775261 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.775236    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.775261 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.775257    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.876331 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.876301    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/edae28b3-e05b-4992-91a1-2cc8c9645f36-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.876331 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.876334    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qscxh\" (UniqueName: \"kubernetes.io/projected/edae28b3-e05b-4992-91a1-2cc8c9645f36-kube-api-access-qscxh\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.876561 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.876355    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.876561 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.876377    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.876561 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.876414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/edae28b3-e05b-4992-91a1-2cc8c9645f36-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.876561 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.876437    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.876561 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.876464    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.877153 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.877128    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.878738 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.878710    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/edae28b3-e05b-4992-91a1-2cc8c9645f36-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.878924 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.878899    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.879146 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.879128    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.879213 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.879144    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/edae28b3-e05b-4992-91a1-2cc8c9645f36-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.884806 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.884783    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qscxh\" (UniqueName: \"kubernetes.io/projected/edae28b3-e05b-4992-91a1-2cc8c9645f36-kube-api-access-qscxh\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.885249 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.885230    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edae28b3-e05b-4992-91a1-2cc8c9645f36-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-75xjq\" (UID: \"edae28b3-e05b-4992-91a1-2cc8c9645f36\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:19.977082 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:19.976992    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:20.107431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:20.107403    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"]
Aug 05 17:23:20.109823 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:23:20.109795    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedae28b3_e05b_4992_91a1_2cc8c9645f36.slice/crio-5dc07c1fff70f18171977822ff3c3efb8f3c0948c8f2d0c26e4990c2a5621e12 WatchSource:0}: Error finding container 5dc07c1fff70f18171977822ff3c3efb8f3c0948c8f2d0c26e4990c2a5621e12: Status 404 returned error can't find the container with id 5dc07c1fff70f18171977822ff3c3efb8f3c0948c8f2d0c26e4990c2a5621e12
Aug 05 17:23:20.172309 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:20.172273    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq" event={"ID":"edae28b3-e05b-4992-91a1-2cc8c9645f36","Type":"ContainerStarted","Data":"5dc07c1fff70f18171977822ff3c3efb8f3c0948c8f2d0c26e4990c2a5621e12"}
Aug 05 17:23:26.916612 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:26.916573    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Aug 05 17:23:26.916972 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:26.916655    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Aug 05 17:23:27.202158 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:27.202075    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq" event={"ID":"edae28b3-e05b-4992-91a1-2cc8c9645f36","Type":"ContainerStarted","Data":"ff9d6946adfb37431b3ee02b093784e93e01f972f4d91d1e9d306d83c33a6e9d"}
Aug 05 17:23:27.202320 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:27.202279    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:27.203802 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:27.203754    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-5677ccb49b-75xjq container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Aug 05 17:23:27.203917 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:27.203817    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq" podUID="edae28b3-e05b-4992-91a1-2cc8c9645f36" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Aug 05 17:23:27.230997 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:27.230933    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq" podStartSLOduration=1.426343097 podStartE2EDuration="8.230912017s" totalImagesPullingTime="6.80456892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:23:27.228915695 +0000 UTC m=+651.928171063" watchObservedRunningTime="2026-08-05 17:23:27.230912017 +0000 UTC m=+651.930167359"
Aug 05 17:23:28.206336 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:28.206307    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-75xjq"
Aug 05 17:23:31.122632 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:31.122599    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cskjl"]
Aug 05 17:23:31.123053 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:31.122759    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-c548b5d7d-cskjl" podUID="d78b0a39-e5fa-4c06-8f20-77c85181d6e2" containerName="authorino" containerID="cri-o://ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc" gracePeriod=30
Aug 05 17:23:31.355755 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:31.355731    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-cskjl"
Aug 05 17:23:31.469008 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:31.468981    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwfs6\" (UniqueName: \"kubernetes.io/projected/d78b0a39-e5fa-4c06-8f20-77c85181d6e2-kube-api-access-jwfs6\") pod \"d78b0a39-e5fa-4c06-8f20-77c85181d6e2\" (UID: \"d78b0a39-e5fa-4c06-8f20-77c85181d6e2\") "
Aug 05 17:23:31.470883 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:31.470861    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d78b0a39-e5fa-4c06-8f20-77c85181d6e2-kube-api-access-jwfs6" (OuterVolumeSpecName: "kube-api-access-jwfs6") pod "d78b0a39-e5fa-4c06-8f20-77c85181d6e2" (UID: "d78b0a39-e5fa-4c06-8f20-77c85181d6e2"). InnerVolumeSpecName "kube-api-access-jwfs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:23:31.569948 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:31.569918    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwfs6\" (UniqueName: \"kubernetes.io/projected/d78b0a39-e5fa-4c06-8f20-77c85181d6e2-kube-api-access-jwfs6\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:23:32.221486 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:32.221451    2574 generic.go:356] "Generic (PLEG): container finished" podID="d78b0a39-e5fa-4c06-8f20-77c85181d6e2" containerID="ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc" exitCode=0
Aug 05 17:23:32.221914 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:32.221488    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-cskjl" event={"ID":"d78b0a39-e5fa-4c06-8f20-77c85181d6e2","Type":"ContainerDied","Data":"ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc"}
Aug 05 17:23:32.221914 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:32.221533    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c548b5d7d-cskjl" event={"ID":"d78b0a39-e5fa-4c06-8f20-77c85181d6e2","Type":"ContainerDied","Data":"8b867569e2b3afbd77017e82c559ae96c20cb7eecf05bfb8ee5545b64b60fe33"}
Aug 05 17:23:32.221914 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:32.221549    2574 scope.go:117] "RemoveContainer" containerID="ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc"
Aug 05 17:23:32.221914 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:32.221501    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c548b5d7d-cskjl"
Aug 05 17:23:32.234276 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:32.234252    2574 scope.go:117] "RemoveContainer" containerID="ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc"
Aug 05 17:23:32.234660 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:23:32.234641    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc\": container with ID starting with ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc not found: ID does not exist" containerID="ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc"
Aug 05 17:23:32.234754 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:32.234671    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc"} err="failed to get container status \"ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc\": rpc error: code = NotFound desc = could not find container \"ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc\": container with ID starting with ecac2dac6108e24981b45b72c130ec0886cc81941dc1e6517b69893e9253a3bc not found: ID does not exist"
Aug 05 17:23:32.243670 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:32.243645    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cskjl"]
Aug 05 17:23:32.247378 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:32.247358    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c548b5d7d-cskjl"]
Aug 05 17:23:33.853808 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:23:33.853754    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d78b0a39-e5fa-4c06-8f20-77c85181d6e2" path="/var/lib/kubelet/pods/d78b0a39-e5fa-4c06-8f20-77c85181d6e2/volumes"
Aug 05 17:27:00.749855 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.749822    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Aug 05 17:27:00.750262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.750114    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d78b0a39-e5fa-4c06-8f20-77c85181d6e2" containerName="authorino"
Aug 05 17:27:00.750262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.750125    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d78b0a39-e5fa-4c06-8f20-77c85181d6e2" containerName="authorino"
Aug 05 17:27:00.750262 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.750178    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d78b0a39-e5fa-4c06-8f20-77c85181d6e2" containerName="authorino"
Aug 05 17:27:00.753333 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.753313    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.755774 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.755749    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisvc-model-fb-opt-125m-route-dc21cb-57cf7736-dockercfg-wb4jg\""
Aug 05 17:27:00.755888 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.755777    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"default-dockercfg-q5sfd\""
Aug 05 17:27:00.755888 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.755756    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"kube-root-ca.crt\""
Aug 05 17:27:00.755888 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.755852    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"odh-kserve-custom-ca-bundle\""
Aug 05 17:27:00.755888 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.755751    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"openshift-service-ca.crt\""
Aug 05 17:27:00.756599 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.756579    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"seaweedfs-s3-creds\""
Aug 05 17:27:00.756729 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.756582    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-63a03ef4\"/\"llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs\""
Aug 05 17:27:00.766372 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.766351    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Aug 05 17:27:00.847958 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.847925    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ab85f08-9b30-460f-b92d-8bc2561535fa-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.848123 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.847969    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.848123 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.848037    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.848123 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.848090    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.848123 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.848118    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.848299 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.848167    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.848299 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.848209    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.848299 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.848227    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6ljj\" (UniqueName: \"kubernetes.io/projected/8ab85f08-9b30-460f-b92d-8bc2561535fa-kube-api-access-c6ljj\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949326 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949301    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949435 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949435 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949358    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949435 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949386    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949556 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949429    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949556 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949458    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949556 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949480    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6ljj\" (UniqueName: \"kubernetes.io/projected/8ab85f08-9b30-460f-b92d-8bc2561535fa-kube-api-access-c6ljj\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949556 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949528    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ab85f08-9b30-460f-b92d-8bc2561535fa-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949749 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949673    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949749 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949732    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949850 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949831    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-home\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.949943 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.949926    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.950098 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.950070    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ab85f08-9b30-460f-b92d-8bc2561535fa-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.951610 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.951591    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-dshm\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.951699 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.951682    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:00.957349 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:00.957327    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6ljj\" (UniqueName: \"kubernetes.io/projected/8ab85f08-9b30-460f-b92d-8bc2561535fa-kube-api-access-c6ljj\") pod \"llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") " pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:01.063394 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:01.063333    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:27:01.191589 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:01.191562    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Aug 05 17:27:01.194311 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:27:01.194287    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab85f08_9b30_460f_b92d_8bc2561535fa.slice/crio-b12cf99829b886fe2b2047971e63d2e4b0c6b53ade959b7963dcf47bd32de483 WatchSource:0}: Error finding container b12cf99829b886fe2b2047971e63d2e4b0c6b53ade959b7963dcf47bd32de483: Status 404 returned error can't find the container with id b12cf99829b886fe2b2047971e63d2e4b0c6b53ade959b7963dcf47bd32de483
Aug 05 17:27:01.196731 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:01.196714    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:27:01.977289 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:01.977247    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8ab85f08-9b30-460f-b92d-8bc2561535fa","Type":"ContainerStarted","Data":"b12cf99829b886fe2b2047971e63d2e4b0c6b53ade959b7963dcf47bd32de483"}
Aug 05 17:27:04.992148 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:04.992116    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8ab85f08-9b30-460f-b92d-8bc2561535fa","Type":"ContainerStarted","Data":"80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69"}
Aug 05 17:27:07.001509 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:07.001428    2574 generic.go:356] "Generic (PLEG): container finished" podID="8ab85f08-9b30-460f-b92d-8bc2561535fa" containerID="80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69" exitCode=0
Aug 05 17:27:07.001509 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:07.001464    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8ab85f08-9b30-460f-b92d-8bc2561535fa","Type":"ContainerDied","Data":"80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69"}
Aug 05 17:27:36.147352 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:36.147318    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8ab85f08-9b30-460f-b92d-8bc2561535fa","Type":"ContainerStarted","Data":"4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab"}
Aug 05 17:27:36.168683 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:27:36.168592    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" podStartSLOduration=2.128307715 podStartE2EDuration="36.168572445s" totalImagesPullingTime="31.991703603s" totalInitContainerRuntime="2.048561127s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:27:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:27:36.167433552 +0000 UTC m=+900.866688893" watchObservedRunningTime="2026-08-05 17:27:36.168572445 +0000 UTC m=+900.867827789"
Aug 05 17:30:07.850087 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:07.850056    2574 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" secret="" err="secret \"llmisvc-model-fb-opt-125m-route-dc21cb-57cf7736-dockercfg-wb4jg\" not found"
Aug 05 17:30:07.901437 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:07.901405    2574 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-63a03ef4/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs: secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:07.901593 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:07.901492    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs podName:8ab85f08-9b30-460f-b92d-8bc2561535fa nodeName:}" failed. No retries permitted until 2026-08-05 17:30:08.401472592 +0000 UTC m=+1053.100727912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs") pod "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa") : secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:08.405282 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:08.405211    2574 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-63a03ef4/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs: secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:08.405448 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:08.405328    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs podName:8ab85f08-9b30-460f-b92d-8bc2561535fa nodeName:}" failed. No retries permitted until 2026-08-05 17:30:09.405310857 +0000 UTC m=+1054.104566177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs") pod "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa") : secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:09.413426 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:09.413390    2574 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-63a03ef4/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs: secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:09.413906 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:09.413476    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs podName:8ab85f08-9b30-460f-b92d-8bc2561535fa nodeName:}" failed. No retries permitted until 2026-08-05 17:30:11.413448541 +0000 UTC m=+1056.112703864 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs") pod "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa") : secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:11.427314 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:11.427270    2574 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-63a03ef4/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs: secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:11.427727 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:11.427348    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs podName:8ab85f08-9b30-460f-b92d-8bc2561535fa nodeName:}" failed. No retries permitted until 2026-08-05 17:30:15.427334151 +0000 UTC m=+1060.126589470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs") pod "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa") : secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:15.459642 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:15.459611    2574 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-63a03ef4/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs: secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:15.459977 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:15.459678    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs podName:8ab85f08-9b30-460f-b92d-8bc2561535fa nodeName:}" failed. No retries permitted until 2026-08-05 17:30:23.459664148 +0000 UTC m=+1068.158919468 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs") pod "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa") : secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:23.524153 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:23.524102    2574 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-63a03ef4/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs: secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:23.524621 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:23.524190    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs podName:8ab85f08-9b30-460f-b92d-8bc2561535fa nodeName:}" failed. No retries permitted until 2026-08-05 17:30:39.524174723 +0000 UTC m=+1084.223430043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs") pod "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa") : secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:24.030385 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:24.030355    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Aug 05 17:30:39.051335 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.051252    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" podUID="8ab85f08-9b30-460f-b92d-8bc2561535fa" containerName="main" containerID="cri-o://4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab" gracePeriod=45
Aug 05 17:30:39.546733 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:39.546709    2574 secret.go:189] Couldn't get secret e2e-test-llm-inference-service-63a03ef4/llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs: secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:39.546828 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:39.546793    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs podName:8ab85f08-9b30-460f-b92d-8bc2561535fa nodeName:}" failed. No retries permitted until 2026-08-05 17:31:11.546768574 +0000 UTC m=+1116.246023894 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs") pod "llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa") : secret "llmisv32fd5468e8138ec3cd54cf7b9c18cfb0-kserve-self-signed-certs" not found
Aug 05 17:30:39.694621 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.694594    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:30:39.849745 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.849661    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-kserve-provision-location\") pod \"8ab85f08-9b30-460f-b92d-8bc2561535fa\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") "
Aug 05 17:30:39.849745 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.849725    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-home\") pod \"8ab85f08-9b30-460f-b92d-8bc2561535fa\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") "
Aug 05 17:30:39.849968 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.849761    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-tmp-dir\") pod \"8ab85f08-9b30-460f-b92d-8bc2561535fa\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") "
Aug 05 17:30:39.849968 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.849797    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6ljj\" (UniqueName: \"kubernetes.io/projected/8ab85f08-9b30-460f-b92d-8bc2561535fa-kube-api-access-c6ljj\") pod \"8ab85f08-9b30-460f-b92d-8bc2561535fa\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") "
Aug 05 17:30:39.849968 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.849846    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs\") pod \"8ab85f08-9b30-460f-b92d-8bc2561535fa\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") "
Aug 05 17:30:39.849968 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.849900    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ab85f08-9b30-460f-b92d-8bc2561535fa-cabundle-cert\") pod \"8ab85f08-9b30-460f-b92d-8bc2561535fa\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") "
Aug 05 17:30:39.849968 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.849922    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-dshm\") pod \"8ab85f08-9b30-460f-b92d-8bc2561535fa\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") "
Aug 05 17:30:39.849968 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.849949    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-model-cache\") pod \"8ab85f08-9b30-460f-b92d-8bc2561535fa\" (UID: \"8ab85f08-9b30-460f-b92d-8bc2561535fa\") "
Aug 05 17:30:39.850425 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.850385    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "8ab85f08-9b30-460f-b92d-8bc2561535fa" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:39.850561 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.850525    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ab85f08-9b30-460f-b92d-8bc2561535fa-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "8ab85f08-9b30-460f-b92d-8bc2561535fa" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:30:39.850857 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.850562    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-home" (OuterVolumeSpecName: "home") pod "8ab85f08-9b30-460f-b92d-8bc2561535fa" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:39.852612 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.852550    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8ab85f08-9b30-460f-b92d-8bc2561535fa" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:30:39.852715 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.852673    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-dshm" (OuterVolumeSpecName: "dshm") pod "8ab85f08-9b30-460f-b92d-8bc2561535fa" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:39.852871 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.852846    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ab85f08-9b30-460f-b92d-8bc2561535fa-kube-api-access-c6ljj" (OuterVolumeSpecName: "kube-api-access-c6ljj") pod "8ab85f08-9b30-460f-b92d-8bc2561535fa" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa"). InnerVolumeSpecName "kube-api-access-c6ljj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:30:39.868324 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.868292    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8ab85f08-9b30-460f-b92d-8bc2561535fa" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:39.876575 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.876549    2574 generic.go:356] "Generic (PLEG): container finished" podID="8ab85f08-9b30-460f-b92d-8bc2561535fa" containerID="4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab" exitCode=0
Aug 05 17:30:39.876658 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.876596    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"
Aug 05 17:30:39.876658 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.876624    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8ab85f08-9b30-460f-b92d-8bc2561535fa","Type":"ContainerDied","Data":"4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab"}
Aug 05 17:30:39.876740 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.876672    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1" event={"ID":"8ab85f08-9b30-460f-b92d-8bc2561535fa","Type":"ContainerDied","Data":"b12cf99829b886fe2b2047971e63d2e4b0c6b53ade959b7963dcf47bd32de483"}
Aug 05 17:30:39.876740 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.876694    2574 scope.go:117] "RemoveContainer" containerID="4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab"
Aug 05 17:30:39.877259 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.877228    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "8ab85f08-9b30-460f-b92d-8bc2561535fa" (UID: "8ab85f08-9b30-460f-b92d-8bc2561535fa"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:30:39.884580 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.884566    2574 scope.go:117] "RemoveContainer" containerID="80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69"
Aug 05 17:30:39.891290 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.891273    2574 scope.go:117] "RemoveContainer" containerID="4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab"
Aug 05 17:30:39.891523 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:39.891503    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab\": container with ID starting with 4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab not found: ID does not exist" containerID="4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab"
Aug 05 17:30:39.891567 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.891531    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab"} err="failed to get container status \"4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab\": rpc error: code = NotFound desc = could not find container \"4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab\": container with ID starting with 4a032bacd4871d3e8e65e93ea9859e38b30d089e298ad9bb4884a209693419ab not found: ID does not exist"
Aug 05 17:30:39.891567 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.891548    2574 scope.go:117] "RemoveContainer" containerID="80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69"
Aug 05 17:30:39.891789 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:30:39.891769    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69\": container with ID starting with 80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69 not found: ID does not exist" containerID="80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69"
Aug 05 17:30:39.891846 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.891798    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69"} err="failed to get container status \"80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69\": rpc error: code = NotFound desc = could not find container \"80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69\": container with ID starting with 80573a11a442bdeab4f250ffa7e2f3d1a6ce3d3e662f6c1ab3702d516e845d69 not found: ID does not exist"
Aug 05 17:30:39.950764 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.950740    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:30:39.950764 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.950762    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:30:39.950894 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.950777    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6ljj\" (UniqueName: \"kubernetes.io/projected/8ab85f08-9b30-460f-b92d-8bc2561535fa-kube-api-access-c6ljj\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:30:39.950894 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.950789    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ab85f08-9b30-460f-b92d-8bc2561535fa-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:30:39.950894 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.950801    2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/8ab85f08-9b30-460f-b92d-8bc2561535fa-cabundle-cert\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:30:39.950894 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.950829    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:30:39.950894 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.950842    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:30:39.950894 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:39.950855    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8ab85f08-9b30-460f-b92d-8bc2561535fa-kserve-provision-location\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:30:40.198064 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:40.198036    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Aug 05 17:30:40.203900 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:40.203876    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-63a03ef4/llmisvc-model-fb-opt-125m-route-dc21cb14-kserve-mn-0-1"]
Aug 05 17:30:41.854748 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:30:41.854711    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ab85f08-9b30-460f-b92d-8bc2561535fa" path="/var/lib/kubelet/pods/8ab85f08-9b30-460f-b92d-8bc2561535fa/volumes"
Aug 05 17:37:04.945312 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.945241    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"]
Aug 05 17:37:04.945733 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.945525    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab85f08-9b30-460f-b92d-8bc2561535fa" containerName="storage-initializer"
Aug 05 17:37:04.945733 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.945535    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab85f08-9b30-460f-b92d-8bc2561535fa" containerName="storage-initializer"
Aug 05 17:37:04.945733 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.945547    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab85f08-9b30-460f-b92d-8bc2561535fa" containerName="main"
Aug 05 17:37:04.945733 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.945552    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab85f08-9b30-460f-b92d-8bc2561535fa" containerName="main"
Aug 05 17:37:04.945733 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.945601    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab85f08-9b30-460f-b92d-8bc2561535fa" containerName="main"
Aug 05 17:37:04.948400 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.948383    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:04.952293 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.952274    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"kube-root-ca.crt\""
Aug 05 17:37:04.952416 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.952371    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"openshift-service-ca.crt\""
Aug 05 17:37:04.952478 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.952419    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"scheduler-v06-pd-migration-test-kserve-self-signed-certs\""
Aug 05 17:37:04.952542 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.952512    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-9c386aa9\"/\"default-dockercfg-gb8zf\""
Aug 05 17:37:04.958505 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:04.958485    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"]
Aug 05 17:37:05.019975 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.019941    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v28nd\" (UniqueName: \"kubernetes.io/projected/d685368f-f268-4d53-b1d2-ae7510065557-kube-api-access-v28nd\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.019975 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.019978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.020214 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.019998    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d685368f-f268-4d53-b1d2-ae7510065557-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.020214 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.020064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.020214 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.020114    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.020214 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.020157    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.121466 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.121441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v28nd\" (UniqueName: \"kubernetes.io/projected/d685368f-f268-4d53-b1d2-ae7510065557-kube-api-access-v28nd\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.121616 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.121481    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.121616 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.121508    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d685368f-f268-4d53-b1d2-ae7510065557-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.121616 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.121538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.121616 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.121571    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.121825 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.121614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.121913 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.121887    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-tmp-dir\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.121986 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.121965    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-home\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.122057 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.122007    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-model-cache\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.123723 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.123703    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-dshm\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.123959 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.123944    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d685368f-f268-4d53-b1d2-ae7510065557-tls-certs\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.131250 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.131222    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v28nd\" (UniqueName: \"kubernetes.io/projected/d685368f-f268-4d53-b1d2-ae7510065557-kube-api-access-v28nd\") pod \"scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") " pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.260400 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.260318    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:05.387337 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.387306    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"]
Aug 05 17:37:05.390333 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:37:05.390308    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd685368f_f268_4d53_b1d2_ae7510065557.slice/crio-b514bcf5a96fd703ce61b8ec0ca14370befed103c2795d2a426d072286ec9195 WatchSource:0}: Error finding container b514bcf5a96fd703ce61b8ec0ca14370befed103c2795d2a426d072286ec9195: Status 404 returned error can't find the container with id b514bcf5a96fd703ce61b8ec0ca14370befed103c2795d2a426d072286ec9195
Aug 05 17:37:05.392092 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:05.392074    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Aug 05 17:37:06.272205 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:06.272169    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr" event={"ID":"d685368f-f268-4d53-b1d2-ae7510065557","Type":"ContainerStarted","Data":"b514bcf5a96fd703ce61b8ec0ca14370befed103c2795d2a426d072286ec9195"}
Aug 05 17:37:07.277090 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:07.277053    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr" event={"ID":"d685368f-f268-4d53-b1d2-ae7510065557","Type":"ContainerStarted","Data":"ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10"}
Aug 05 17:37:07.300228 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:07.300183    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr" podStartSLOduration=1.796610114 podStartE2EDuration="3.300167424s" totalImagesPullingTime="1.50355731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:37:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:37:07.298912176 +0000 UTC m=+1471.998167529" watchObservedRunningTime="2026-08-05 17:37:07.300167424 +0000 UTC m=+1471.999422766"
Aug 05 17:37:08.260570 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:08.260531    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:15.260805 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:15.260766    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:15.272863 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:15.272830    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:37:15.318345 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:37:15.318318    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:38:39.061805 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.061771    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"]
Aug 05 17:38:39.071669 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:38:39.071621    2574 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:38:39.071669 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:38:39.071669 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:38:39.071669 ip-10-0-141-246 kubenswrapper[2574]:  > containerID="ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10" cmd=["/bin/sleep","15"]
Aug 05 17:38:39.071974 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.071692    2574 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr" message="" err=<
Aug 05 17:38:39.071974 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:38:39.071974 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:38:39.071974 ip-10-0-141-246 kubenswrapper[2574]:  >
Aug 05 17:38:39.071974 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:38:39.071709    2574 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:38:39.071974 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:38:39.071974 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:38:39.071974 ip-10-0-141-246 kubenswrapper[2574]:  > pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr" podUID="d685368f-f268-4d53-b1d2-ae7510065557" containerName="main" containerID="cri-o://ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10"
Aug 05 17:38:39.071974 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.071764    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr" podUID="d685368f-f268-4d53-b1d2-ae7510065557" containerName="main" containerID="cri-o://ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10" gracePeriod=60
Aug 05 17:38:39.308750 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.308712    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr" podUID="d685368f-f268-4d53-b1d2-ae7510065557" containerName="main" probeResult="failure" output="Get \"https://10.134.0.24:8000/health\": dial tcp 10.134.0.24:8000: connect: connection refused"
Aug 05 17:38:39.407676 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.407656    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:38:39.446535 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.446514    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-dshm\") pod \"d685368f-f268-4d53-b1d2-ae7510065557\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") "
Aug 05 17:38:39.446643 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.446544    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-home\") pod \"d685368f-f268-4d53-b1d2-ae7510065557\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") "
Aug 05 17:38:39.446643 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.446587    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v28nd\" (UniqueName: \"kubernetes.io/projected/d685368f-f268-4d53-b1d2-ae7510065557-kube-api-access-v28nd\") pod \"d685368f-f268-4d53-b1d2-ae7510065557\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") "
Aug 05 17:38:39.446643 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.446632    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d685368f-f268-4d53-b1d2-ae7510065557-tls-certs\") pod \"d685368f-f268-4d53-b1d2-ae7510065557\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") "
Aug 05 17:38:39.446805 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.446662    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-tmp-dir\") pod \"d685368f-f268-4d53-b1d2-ae7510065557\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") "
Aug 05 17:38:39.446805 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.446720    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-model-cache\") pod \"d685368f-f268-4d53-b1d2-ae7510065557\" (UID: \"d685368f-f268-4d53-b1d2-ae7510065557\") "
Aug 05 17:38:39.446910 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.446804    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-home" (OuterVolumeSpecName: "home") pod "d685368f-f268-4d53-b1d2-ae7510065557" (UID: "d685368f-f268-4d53-b1d2-ae7510065557"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:38:39.446970 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.446922    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d685368f-f268-4d53-b1d2-ae7510065557" (UID: "d685368f-f268-4d53-b1d2-ae7510065557"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:38:39.447061 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.447036    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-model-cache" (OuterVolumeSpecName: "model-cache") pod "d685368f-f268-4d53-b1d2-ae7510065557" (UID: "d685368f-f268-4d53-b1d2-ae7510065557"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:38:39.447121 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.447060    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:38:39.447121 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.447080    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:38:39.448650 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.448627    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-dshm" (OuterVolumeSpecName: "dshm") pod "d685368f-f268-4d53-b1d2-ae7510065557" (UID: "d685368f-f268-4d53-b1d2-ae7510065557"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:38:39.448759 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.448688    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d685368f-f268-4d53-b1d2-ae7510065557-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d685368f-f268-4d53-b1d2-ae7510065557" (UID: "d685368f-f268-4d53-b1d2-ae7510065557"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:38:39.448820 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.448792    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d685368f-f268-4d53-b1d2-ae7510065557-kube-api-access-v28nd" (OuterVolumeSpecName: "kube-api-access-v28nd") pod "d685368f-f268-4d53-b1d2-ae7510065557" (UID: "d685368f-f268-4d53-b1d2-ae7510065557"). InnerVolumeSpecName "kube-api-access-v28nd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:38:39.548139 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.548112    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d685368f-f268-4d53-b1d2-ae7510065557-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:38:39.548139 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.548134    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:38:39.548139 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.548142    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d685368f-f268-4d53-b1d2-ae7510065557-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:38:39.548339 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.548151    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v28nd\" (UniqueName: \"kubernetes.io/projected/d685368f-f268-4d53-b1d2-ae7510065557-kube-api-access-v28nd\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:38:39.623453 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.623386    2574 generic.go:356] "Generic (PLEG): container finished" podID="d685368f-f268-4d53-b1d2-ae7510065557" containerID="ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10" exitCode=0
Aug 05 17:38:39.623453 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.623429    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"
Aug 05 17:38:39.623607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.623475    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr" event={"ID":"d685368f-f268-4d53-b1d2-ae7510065557","Type":"ContainerDied","Data":"ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10"}
Aug 05 17:38:39.623607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.623526    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr" event={"ID":"d685368f-f268-4d53-b1d2-ae7510065557","Type":"ContainerDied","Data":"b514bcf5a96fd703ce61b8ec0ca14370befed103c2795d2a426d072286ec9195"}
Aug 05 17:38:39.623607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.623549    2574 scope.go:117] "RemoveContainer" containerID="ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10"
Aug 05 17:38:39.632279 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.632260    2574 scope.go:117] "RemoveContainer" containerID="ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10"
Aug 05 17:38:39.634071 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:38:39.634038    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10\": container with ID starting with ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10 not found: ID does not exist" containerID="ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10"
Aug 05 17:38:39.634071 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.634067    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10"} err="failed to get container status \"ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10\": rpc error: code = NotFound desc = could not find container \"ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10\": container with ID starting with ea1728bcc7d91a99dadbe001ac31936b29be64dce2090cd3af9cf82212028d10 not found: ID does not exist"
Aug 05 17:38:39.647071 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.647049    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"]
Aug 05 17:38:39.653311 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.653285    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-9c386aa9/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-shnkr"]
Aug 05 17:38:39.854686 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:38:39.854652    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d685368f-f268-4d53-b1d2-ae7510065557" path="/var/lib/kubelet/pods/d685368f-f268-4d53-b1d2-ae7510065557/volumes"
Aug 05 17:40:47.523730 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.523698    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"]
Aug 05 17:40:47.524180 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.523980    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d685368f-f268-4d53-b1d2-ae7510065557" containerName="main"
Aug 05 17:40:47.524180 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.523991    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d685368f-f268-4d53-b1d2-ae7510065557" containerName="main"
Aug 05 17:40:47.524180 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.524055    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d685368f-f268-4d53-b1d2-ae7510065557" containerName="main"
Aug 05 17:40:47.526814 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.526799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.529715 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.529692    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"seaweedfs-s3-creds\""
Aug 05 17:40:47.529874 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.529722    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"kube-root-ca.crt\""
Aug 05 17:40:47.530133 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.530116    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"odh-kserve-custom-ca-bundle\""
Aug 05 17:40:47.534982 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.534965    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"default-dockercfg-qbppr\""
Aug 05 17:40:47.535283 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.535256    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"tokenizer-clean-path-test-kserve-self-signed-certs\""
Aug 05 17:40:47.535381 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.535286    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-cf47448b\"/\"openshift-service-ca.crt\""
Aug 05 17:40:47.542235 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.542211    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"]
Aug 05 17:40:47.613408 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.613377    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvp5v\" (UniqueName: \"kubernetes.io/projected/767ad469-123f-45b3-ba65-7e19e8fd4e22-kube-api-access-zvp5v\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.613408 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.613408    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-dshm\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.613607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.613431    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-home\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.613607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.613457    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/767ad469-123f-45b3-ba65-7e19e8fd4e22-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.613607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.613483    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.613607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.613507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-model-cache\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.613607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.613525    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/767ad469-123f-45b3-ba65-7e19e8fd4e22-tls-certs\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.613607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.613551    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714359 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714335    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvp5v\" (UniqueName: \"kubernetes.io/projected/767ad469-123f-45b3-ba65-7e19e8fd4e22-kube-api-access-zvp5v\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714462 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714362    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-dshm\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714462 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714383    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-home\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714462 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714405    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/767ad469-123f-45b3-ba65-7e19e8fd4e22-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714462 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714424    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714462 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714455    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-model-cache\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714704 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714576    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/767ad469-123f-45b3-ba65-7e19e8fd4e22-tls-certs\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714704 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714616    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714875 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714853    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-home\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714938 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714875    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-model-cache\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714993 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714953    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-kserve-provision-location\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.714993 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.714976    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-tmp-dir\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.715190 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.715173    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/767ad469-123f-45b3-ba65-7e19e8fd4e22-cabundle-cert\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.716597 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.716572    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-dshm\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.717035 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.717000    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/767ad469-123f-45b3-ba65-7e19e8fd4e22-tls-certs\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.736576 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.736553    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvp5v\" (UniqueName: \"kubernetes.io/projected/767ad469-123f-45b3-ba65-7e19e8fd4e22-kube-api-access-zvp5v\") pod \"tokenizer-clean-path-test-kserve-58cc87dd64-v77nr\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") " pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:47.836567 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:47.836506    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:48.008007 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:48.007971    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"]
Aug 05 17:40:48.011675 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:40:48.011640    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod767ad469_123f_45b3_ba65_7e19e8fd4e22.slice/crio-495c522d4ec05c9f4b9341653fdb003d0be9f906631656f6a9491c175eebe658 WatchSource:0}: Error finding container 495c522d4ec05c9f4b9341653fdb003d0be9f906631656f6a9491c175eebe658: Status 404 returned error can't find the container with id 495c522d4ec05c9f4b9341653fdb003d0be9f906631656f6a9491c175eebe658
Aug 05 17:40:48.098184 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:48.098117    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" event={"ID":"767ad469-123f-45b3-ba65-7e19e8fd4e22","Type":"ContainerStarted","Data":"a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3"}
Aug 05 17:40:48.098315 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:48.098188    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" event={"ID":"767ad469-123f-45b3-ba65-7e19e8fd4e22","Type":"ContainerStarted","Data":"495c522d4ec05c9f4b9341653fdb003d0be9f906631656f6a9491c175eebe658"}
Aug 05 17:40:51.111062 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:51.111004    2574 generic.go:356] "Generic (PLEG): container finished" podID="767ad469-123f-45b3-ba65-7e19e8fd4e22" containerID="a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3" exitCode=0
Aug 05 17:40:51.111415 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:51.111085    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" event={"ID":"767ad469-123f-45b3-ba65-7e19e8fd4e22","Type":"ContainerDied","Data":"a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3"}
Aug 05 17:40:52.116760 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:52.116721    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" event={"ID":"767ad469-123f-45b3-ba65-7e19e8fd4e22","Type":"ContainerStarted","Data":"eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a"}
Aug 05 17:40:52.836856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:52.836823    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:57.837301 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:57.837217    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:57.854336 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:57.854310    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:40:57.878413 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:57.878367    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" podStartSLOduration=8.825442644 podStartE2EDuration="10.878353489s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.052910845s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:40:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:40:52.148611471 +0000 UTC m=+1696.847866813" watchObservedRunningTime="2026-08-05 17:40:57.878353489 +0000 UTC m=+1702.577608830"
Aug 05 17:40:58.149741 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:40:58.149664    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:42:36.703725 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:36.703690    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"]
Aug 05 17:42:36.712260 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:42:36.712230    2574 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:42:36.712260 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:42:36.712260 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:42:36.712260 ip-10-0-141-246 kubenswrapper[2574]:  > containerID="eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a" cmd=["/bin/sleep","15"]
Aug 05 17:42:36.712461 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:36.712298    2574 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" message="" err=<
Aug 05 17:42:36.712461 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:42:36.712461 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:42:36.712461 ip-10-0-141-246 kubenswrapper[2574]:  >
Aug 05 17:42:36.712461 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:42:36.712322    2574 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:42:36.712461 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:42:36.712461 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:42:36.712461 ip-10-0-141-246 kubenswrapper[2574]:  > pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" podUID="767ad469-123f-45b3-ba65-7e19e8fd4e22" containerName="main" containerID="cri-o://eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a"
Aug 05 17:42:36.712461 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:36.712356    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" podUID="767ad469-123f-45b3-ba65-7e19e8fd4e22" containerName="main" containerID="cri-o://eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a" gracePeriod=60
Aug 05 17:42:37.053009 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.052985    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:42:37.147523 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147491    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-dshm\") pod \"767ad469-123f-45b3-ba65-7e19e8fd4e22\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") "
Aug 05 17:42:37.147523 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147527    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-model-cache\") pod \"767ad469-123f-45b3-ba65-7e19e8fd4e22\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") "
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147554    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-kserve-provision-location\") pod \"767ad469-123f-45b3-ba65-7e19e8fd4e22\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") "
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147612    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/767ad469-123f-45b3-ba65-7e19e8fd4e22-tls-certs\") pod \"767ad469-123f-45b3-ba65-7e19e8fd4e22\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") "
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147636    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-tmp-dir\") pod \"767ad469-123f-45b3-ba65-7e19e8fd4e22\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") "
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147677    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvp5v\" (UniqueName: \"kubernetes.io/projected/767ad469-123f-45b3-ba65-7e19e8fd4e22-kube-api-access-zvp5v\") pod \"767ad469-123f-45b3-ba65-7e19e8fd4e22\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") "
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147711    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-home\") pod \"767ad469-123f-45b3-ba65-7e19e8fd4e22\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") "
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147741    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/767ad469-123f-45b3-ba65-7e19e8fd4e22-cabundle-cert\") pod \"767ad469-123f-45b3-ba65-7e19e8fd4e22\" (UID: \"767ad469-123f-45b3-ba65-7e19e8fd4e22\") "
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147793    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-model-cache" (OuterVolumeSpecName: "model-cache") pod "767ad469-123f-45b3-ba65-7e19e8fd4e22" (UID: "767ad469-123f-45b3-ba65-7e19e8fd4e22"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.147889    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "767ad469-123f-45b3-ba65-7e19e8fd4e22" (UID: "767ad469-123f-45b3-ba65-7e19e8fd4e22"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.148043    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.148064    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.148098    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-home" (OuterVolumeSpecName: "home") pod "767ad469-123f-45b3-ba65-7e19e8fd4e22" (UID: "767ad469-123f-45b3-ba65-7e19e8fd4e22"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:37.148208 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.148127    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/767ad469-123f-45b3-ba65-7e19e8fd4e22-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "767ad469-123f-45b3-ba65-7e19e8fd4e22" (UID: "767ad469-123f-45b3-ba65-7e19e8fd4e22"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:42:37.150330 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.150238    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/767ad469-123f-45b3-ba65-7e19e8fd4e22-kube-api-access-zvp5v" (OuterVolumeSpecName: "kube-api-access-zvp5v") pod "767ad469-123f-45b3-ba65-7e19e8fd4e22" (UID: "767ad469-123f-45b3-ba65-7e19e8fd4e22"). InnerVolumeSpecName "kube-api-access-zvp5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:42:37.150330 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.150253    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/767ad469-123f-45b3-ba65-7e19e8fd4e22-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "767ad469-123f-45b3-ba65-7e19e8fd4e22" (UID: "767ad469-123f-45b3-ba65-7e19e8fd4e22"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:42:37.150330 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.150255    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-dshm" (OuterVolumeSpecName: "dshm") pod "767ad469-123f-45b3-ba65-7e19e8fd4e22" (UID: "767ad469-123f-45b3-ba65-7e19e8fd4e22"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:37.168222 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.168197    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "767ad469-123f-45b3-ba65-7e19e8fd4e22" (UID: "767ad469-123f-45b3-ba65-7e19e8fd4e22"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:42:37.249097 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.249045    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/767ad469-123f-45b3-ba65-7e19e8fd4e22-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:42:37.249097 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.249066    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvp5v\" (UniqueName: \"kubernetes.io/projected/767ad469-123f-45b3-ba65-7e19e8fd4e22-kube-api-access-zvp5v\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:42:37.249097 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.249077    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:42:37.249097 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.249086    2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/767ad469-123f-45b3-ba65-7e19e8fd4e22-cabundle-cert\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:42:37.249097 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.249094    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:42:37.249338 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.249102    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/767ad469-123f-45b3-ba65-7e19e8fd4e22-kserve-provision-location\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:42:37.523902 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.523824    2574 generic.go:356] "Generic (PLEG): container finished" podID="767ad469-123f-45b3-ba65-7e19e8fd4e22" containerID="eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a" exitCode=0
Aug 05 17:42:37.523902 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.523872    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"
Aug 05 17:42:37.523902 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.523886    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" event={"ID":"767ad469-123f-45b3-ba65-7e19e8fd4e22","Type":"ContainerDied","Data":"eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a"}
Aug 05 17:42:37.524153 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.523958    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr" event={"ID":"767ad469-123f-45b3-ba65-7e19e8fd4e22","Type":"ContainerDied","Data":"495c522d4ec05c9f4b9341653fdb003d0be9f906631656f6a9491c175eebe658"}
Aug 05 17:42:37.524153 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.523981    2574 scope.go:117] "RemoveContainer" containerID="eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a"
Aug 05 17:42:37.532592 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.532576    2574 scope.go:117] "RemoveContainer" containerID="a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3"
Aug 05 17:42:37.539605 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.539587    2574 scope.go:117] "RemoveContainer" containerID="eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a"
Aug 05 17:42:37.539830 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:42:37.539811    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a\": container with ID starting with eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a not found: ID does not exist" containerID="eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a"
Aug 05 17:42:37.539878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.539839    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a"} err="failed to get container status \"eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a\": rpc error: code = NotFound desc = could not find container \"eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a\": container with ID starting with eb93b531a505f2e6993ebae06840fa982346fffcff6dab2c74d1d8d706b2c69a not found: ID does not exist"
Aug 05 17:42:37.539878 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.539856    2574 scope.go:117] "RemoveContainer" containerID="a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3"
Aug 05 17:42:37.540095 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:42:37.540078    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3\": container with ID starting with a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3 not found: ID does not exist" containerID="a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3"
Aug 05 17:42:37.540134 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.540101    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3"} err="failed to get container status \"a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3\": rpc error: code = NotFound desc = could not find container \"a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3\": container with ID starting with a9361f304306dfda85bb9b7518ab665ddd2978f542ef00795b04ccd2db599ac3 not found: ID does not exist"
Aug 05 17:42:37.545590 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.545569    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"]
Aug 05 17:42:37.551702 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.551682    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-cf47448b/tokenizer-clean-path-test-kserve-58cc87dd64-v77nr"]
Aug 05 17:42:37.854496 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:42:37.854419    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="767ad469-123f-45b3-ba65-7e19e8fd4e22" path="/var/lib/kubelet/pods/767ad469-123f-45b3-ba65-7e19e8fd4e22/volumes"
Aug 05 17:43:20.638860 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.638828    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"]
Aug 05 17:43:20.639486 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.639152    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="767ad469-123f-45b3-ba65-7e19e8fd4e22" containerName="storage-initializer"
Aug 05 17:43:20.639486 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.639164    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="767ad469-123f-45b3-ba65-7e19e8fd4e22" containerName="storage-initializer"
Aug 05 17:43:20.639486 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.639175    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="767ad469-123f-45b3-ba65-7e19e8fd4e22" containerName="main"
Aug 05 17:43:20.639486 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.639182    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="767ad469-123f-45b3-ba65-7e19e8fd4e22" containerName="main"
Aug 05 17:43:20.639486 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.639241    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="767ad469-123f-45b3-ba65-7e19e8fd4e22" containerName="main"
Aug 05 17:43:20.642060 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.642042    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.644259 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.644234    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"seaweedfs-s3-creds\""
Aug 05 17:43:20.644405 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.644392    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"default-dockercfg-jwjml\""
Aug 05 17:43:20.644479 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.644468    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"openshift-service-ca.crt\""
Aug 05 17:43:20.645117 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.645094    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"odh-kserve-custom-ca-bundle\""
Aug 05 17:43:20.645219 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.645159    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"kube-root-ca.crt\""
Aug 05 17:43:20.645219 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.645168    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-312caf2d\"/\"tokenizer-migration-path-test-kserve-self-signed-certs\""
Aug 05 17:43:20.657415 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.657324    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"]
Aug 05 17:43:20.773037 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.772991    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-dshm\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.773179 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.773044    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.773179 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.773090    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tls-certs\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.773179 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.773110    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.773294 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.773177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk7wt\" (UniqueName: \"kubernetes.io/projected/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kube-api-access-hk7wt\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.773294 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.773213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-home\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.773294 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.773230    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.773294 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.773252    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-model-cache\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874103 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874056    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-dshm\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874270 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874109    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874270 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874153    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tls-certs\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874270 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874270 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874239    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk7wt\" (UniqueName: \"kubernetes.io/projected/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kube-api-access-hk7wt\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874270 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874267    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-home\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874538 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874291    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874538 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874321    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-model-cache\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874766 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874741    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-model-cache\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.874949 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.874799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tmp-dir\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.875342 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.875322    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-home\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.875465 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.875418    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-cabundle-cert\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.875886 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.875858    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kserve-provision-location\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.877168 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.877148    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-dshm\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.877810 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.877791    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tls-certs\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.887699 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.887677    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk7wt\" (UniqueName: \"kubernetes.io/projected/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kube-api-access-hk7wt\") pod \"tokenizer-migration-path-test-kserve-68595dd9c-5jmt2\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") " pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:20.952155 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:20.952130    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:21.082868 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:21.082788    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"]
Aug 05 17:43:21.084734 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:43:21.084711    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb9feb3_e5e4_4ff3_ae02_3ec6501aacb6.slice/crio-4c1b7103e62cad17c3db3f04f2a449fac7d0df9896a347e61a6d25c26075d2f7 WatchSource:0}: Error finding container 4c1b7103e62cad17c3db3f04f2a449fac7d0df9896a347e61a6d25c26075d2f7: Status 404 returned error can't find the container with id 4c1b7103e62cad17c3db3f04f2a449fac7d0df9896a347e61a6d25c26075d2f7
Aug 05 17:43:21.691124 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:21.691082    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" event={"ID":"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6","Type":"ContainerStarted","Data":"d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0"}
Aug 05 17:43:21.691497 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:21.691145    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" event={"ID":"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6","Type":"ContainerStarted","Data":"4c1b7103e62cad17c3db3f04f2a449fac7d0df9896a347e61a6d25c26075d2f7"}
Aug 05 17:43:23.702033 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:23.701987    2574 generic.go:356] "Generic (PLEG): container finished" podID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" containerID="d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0" exitCode=0
Aug 05 17:43:23.702465 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:23.702056    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" event={"ID":"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6","Type":"ContainerDied","Data":"d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0"}
Aug 05 17:43:24.707462 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:24.707427    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" event={"ID":"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6","Type":"ContainerStarted","Data":"717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5"}
Aug 05 17:43:24.730576 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:24.730534    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" podStartSLOduration=2.693414287 podStartE2EDuration="4.730519708s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.037105421s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:43:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:43:24.72868626 +0000 UTC m=+1849.427941603" watchObservedRunningTime="2026-08-05 17:43:24.730519708 +0000 UTC m=+1849.429775102"
Aug 05 17:43:24.952683 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:24.952645    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:30.953074 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:30.953026    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:30.965436 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:30.965411    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:43:31.744770 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:43:31.744741    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:44:02.971543 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:02.971461    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"]
Aug 05 17:44:02.983609 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:44:02.983571    2574 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:44:02.983609 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:44:02.983609 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:44:02.983609 ip-10-0-141-246 kubenswrapper[2574]:  > containerID="717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5" cmd=["/bin/sleep","15"]
Aug 05 17:44:02.983851 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:02.983632    2574 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" message="" err=<
Aug 05 17:44:02.983851 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:44:02.983851 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:44:02.983851 ip-10-0-141-246 kubenswrapper[2574]:  >
Aug 05 17:44:02.983851 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:44:02.983648    2574 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:44:02.983851 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:44:02.983851 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:44:02.983851 ip-10-0-141-246 kubenswrapper[2574]:  > pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" podUID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" containerName="main" containerID="cri-o://717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5"
Aug 05 17:44:02.983851 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:02.983701    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" podUID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" containerName="main" containerID="cri-o://717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5" gracePeriod=60
Aug 05 17:44:03.325954 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.325932    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:44:03.400974 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.400948    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-dshm\") pod \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") "
Aug 05 17:44:03.401148 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.400980    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hk7wt\" (UniqueName: \"kubernetes.io/projected/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kube-api-access-hk7wt\") pod \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") "
Aug 05 17:44:03.401148 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.400998    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-model-cache\") pod \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") "
Aug 05 17:44:03.401148 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401076    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-home\") pod \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") "
Aug 05 17:44:03.401148 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401098    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-cabundle-cert\") pod \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") "
Aug 05 17:44:03.401148 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401143    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tls-certs\") pod \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") "
Aug 05 17:44:03.401431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401164    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tmp-dir\") pod \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") "
Aug 05 17:44:03.401431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401189    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kserve-provision-location\") pod \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\" (UID: \"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6\") "
Aug 05 17:44:03.401431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401329    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-home" (OuterVolumeSpecName: "home") pod "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" (UID: "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:44:03.401431 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401344    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-model-cache" (OuterVolumeSpecName: "model-cache") pod "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" (UID: "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:44:03.401641 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401487    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:44:03.401641 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401489    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" (UID: "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:44:03.401641 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401505    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:44:03.401790 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.401672    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" (UID: "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:44:03.403395 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.403343    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" (UID: "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:44:03.403500 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.403401    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kube-api-access-hk7wt" (OuterVolumeSpecName: "kube-api-access-hk7wt") pod "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" (UID: "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6"). InnerVolumeSpecName "kube-api-access-hk7wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:44:03.403500 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.403411    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-dshm" (OuterVolumeSpecName: "dshm") pod "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" (UID: "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:44:03.422100 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.422079    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" (UID: "2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:44:03.502642 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.502582    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:44:03.502642 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.502603    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:44:03.502642 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.502614    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kserve-provision-location\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:44:03.502642 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.502622    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:44:03.502642 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.502631    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hk7wt\" (UniqueName: \"kubernetes.io/projected/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-kube-api-access-hk7wt\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:44:03.502642 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.502639    2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6-cabundle-cert\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:44:03.849754 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.849672    2574 generic.go:356] "Generic (PLEG): container finished" podID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" containerID="717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5" exitCode=0
Aug 05 17:44:03.849890 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.849779    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"
Aug 05 17:44:03.854283 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.854252    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" event={"ID":"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6","Type":"ContainerDied","Data":"717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5"}
Aug 05 17:44:03.854410 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.854306    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2" event={"ID":"2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6","Type":"ContainerDied","Data":"4c1b7103e62cad17c3db3f04f2a449fac7d0df9896a347e61a6d25c26075d2f7"}
Aug 05 17:44:03.854410 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.854338    2574 scope.go:117] "RemoveContainer" containerID="717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5"
Aug 05 17:44:03.863633 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.863618    2574 scope.go:117] "RemoveContainer" containerID="d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0"
Aug 05 17:44:03.870983 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.870966    2574 scope.go:117] "RemoveContainer" containerID="717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5"
Aug 05 17:44:03.871290 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:44:03.871273    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5\": container with ID starting with 717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5 not found: ID does not exist" containerID="717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5"
Aug 05 17:44:03.871339 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.871298    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5"} err="failed to get container status \"717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5\": rpc error: code = NotFound desc = could not find container \"717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5\": container with ID starting with 717c73e507c3aed9a7ec9e1f21f7659b0c4c80a0e6f3beff2817fd5c6c7960b5 not found: ID does not exist"
Aug 05 17:44:03.871339 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.871314    2574 scope.go:117] "RemoveContainer" containerID="d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0"
Aug 05 17:44:03.871565 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:44:03.871548    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0\": container with ID starting with d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0 not found: ID does not exist" containerID="d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0"
Aug 05 17:44:03.871613 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.871571    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0"} err="failed to get container status \"d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0\": rpc error: code = NotFound desc = could not find container \"d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0\": container with ID starting with d647e2cc2f3052e417a763c2eb9ac1a9eec872cde088d37b04b5c410109c5cd0 not found: ID does not exist"
Aug 05 17:44:03.875573 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.875553    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"]
Aug 05 17:44:03.885832 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:03.885809    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-312caf2d/tokenizer-migration-path-test-kserve-68595dd9c-5jmt2"]
Aug 05 17:44:05.855792 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:44:05.855754    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" path="/var/lib/kubelet/pods/2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6/volumes"
Aug 05 17:45:17.102167 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.102135    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"]
Aug 05 17:45:17.102760 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.102533    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" containerName="storage-initializer"
Aug 05 17:45:17.102760 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.102549    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" containerName="storage-initializer"
Aug 05 17:45:17.102760 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.102575    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" containerName="main"
Aug 05 17:45:17.102760 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.102584    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" containerName="main"
Aug 05 17:45:17.102760 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.102671    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb9feb3-e5e4-4ff3-ae02-3ec6501aacb6" containerName="main"
Aug 05 17:45:17.105629 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.105607    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.109690 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.109669    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"seaweedfs-s3-creds\""
Aug 05 17:45:17.110183 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.110167    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"tokenizer-template-args-test-kserve-self-signed-certs\""
Aug 05 17:45:17.110447 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.110434    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"default-dockercfg-2nw7j\""
Aug 05 17:45:17.110819 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.110798    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"odh-kserve-custom-ca-bundle\""
Aug 05 17:45:17.110920 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.110829    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"kube-root-ca.crt\""
Aug 05 17:45:17.110920 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.110834    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-0eb24714\"/\"openshift-service-ca.crt\""
Aug 05 17:45:17.120050 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.120006    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"]
Aug 05 17:45:17.273720 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.273684    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-dshm\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.273886 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.273734    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee62092a-fc94-47f6-bef3-8ac32b125367-tls-certs\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.273886 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.273760    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.273886 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.273793    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-tmp-dir\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.273886 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.273817    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv4zc\" (UniqueName: \"kubernetes.io/projected/ee62092a-fc94-47f6-bef3-8ac32b125367-kube-api-access-tv4zc\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.273886 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.273836    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-model-cache\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.274074 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.273891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee62092a-fc94-47f6-bef3-8ac32b125367-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.274074 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.273950    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-home\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.374617 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.374533    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee62092a-fc94-47f6-bef3-8ac32b125367-tls-certs\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.374617 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.374576    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.374617 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.374604    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-tmp-dir\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.374827 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.374625    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv4zc\" (UniqueName: \"kubernetes.io/projected/ee62092a-fc94-47f6-bef3-8ac32b125367-kube-api-access-tv4zc\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.374827 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.374646    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-model-cache\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.374827 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.374685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee62092a-fc94-47f6-bef3-8ac32b125367-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.374827 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.374720    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-home\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.374827 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.374746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-dshm\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.375067 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.375045    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-kserve-provision-location\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.375137 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.375064    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-tmp-dir\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.375240 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.375215    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-model-cache\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.375299 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.375242    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-home\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.375581 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.375563    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee62092a-fc94-47f6-bef3-8ac32b125367-cabundle-cert\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.376933 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.376908    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-dshm\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.377219 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.377202    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee62092a-fc94-47f6-bef3-8ac32b125367-tls-certs\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.391384 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.391356    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv4zc\" (UniqueName: \"kubernetes.io/projected/ee62092a-fc94-47f6-bef3-8ac32b125367-kube-api-access-tv4zc\") pod \"tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") " pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.416424 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.416387    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:17.561427 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:17.560716    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"]
Aug 05 17:45:18.121721 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:18.121685    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" event={"ID":"ee62092a-fc94-47f6-bef3-8ac32b125367","Type":"ContainerStarted","Data":"dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4"}
Aug 05 17:45:18.122118 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:18.121747    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" event={"ID":"ee62092a-fc94-47f6-bef3-8ac32b125367","Type":"ContainerStarted","Data":"408c37449a0b7bb4aa4f08d5071fad4dc8892757f10d8880f9f54e96113a425a"}
Aug 05 17:45:20.131670 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:20.131591    2574 generic.go:356] "Generic (PLEG): container finished" podID="ee62092a-fc94-47f6-bef3-8ac32b125367" containerID="dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4" exitCode=0
Aug 05 17:45:20.131670 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:20.131663    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" event={"ID":"ee62092a-fc94-47f6-bef3-8ac32b125367","Type":"ContainerDied","Data":"dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4"}
Aug 05 17:45:21.137402 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:21.137367    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" event={"ID":"ee62092a-fc94-47f6-bef3-8ac32b125367","Type":"ContainerStarted","Data":"724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1"}
Aug 05 17:45:21.173150 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:21.173090    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" podStartSLOduration=2.080049736 podStartE2EDuration="4.173073865s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.093024129s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:45:21.171762198 +0000 UTC m=+1965.871017544" watchObservedRunningTime="2026-08-05 17:45:21.173073865 +0000 UTC m=+1965.872329210"
Aug 05 17:45:21.416735 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:21.416694    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:27.417096 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:27.417061    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:27.429428 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:27.429405    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:28.173048 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:28.173003    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:56.775293 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:56.775260    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"]
Aug 05 17:45:56.784653 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:45:56.784622    2574 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:45:56.784653 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:45:56.784653 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:45:56.784653 ip-10-0-141-246 kubenswrapper[2574]:  > containerID="724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1" cmd=["/bin/sleep","15"]
Aug 05 17:45:56.784911 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:56.784668    2574 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" message="" err=<
Aug 05 17:45:56.784911 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:45:56.784911 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:45:56.784911 ip-10-0-141-246 kubenswrapper[2574]:  >
Aug 05 17:45:56.784911 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:45:56.784680    2574 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:45:56.784911 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:45:56.784911 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:45:56.784911 ip-10-0-141-246 kubenswrapper[2574]:  > pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" podUID="ee62092a-fc94-47f6-bef3-8ac32b125367" containerName="main" containerID="cri-o://724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1"
Aug 05 17:45:56.784911 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:56.784713    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" podUID="ee62092a-fc94-47f6-bef3-8ac32b125367" containerName="main" containerID="cri-o://724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1" gracePeriod=60
Aug 05 17:45:57.024049 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.024026    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:57.075399 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075346    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tv4zc\" (UniqueName: \"kubernetes.io/projected/ee62092a-fc94-47f6-bef3-8ac32b125367-kube-api-access-tv4zc\") pod \"ee62092a-fc94-47f6-bef3-8ac32b125367\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") "
Aug 05 17:45:57.075399 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075384    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-home\") pod \"ee62092a-fc94-47f6-bef3-8ac32b125367\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") "
Aug 05 17:45:57.075562 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075413    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-tmp-dir\") pod \"ee62092a-fc94-47f6-bef3-8ac32b125367\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") "
Aug 05 17:45:57.075562 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075440    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-dshm\") pod \"ee62092a-fc94-47f6-bef3-8ac32b125367\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") "
Aug 05 17:45:57.075562 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075462    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee62092a-fc94-47f6-bef3-8ac32b125367-tls-certs\") pod \"ee62092a-fc94-47f6-bef3-8ac32b125367\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") "
Aug 05 17:45:57.075562 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075523    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee62092a-fc94-47f6-bef3-8ac32b125367-cabundle-cert\") pod \"ee62092a-fc94-47f6-bef3-8ac32b125367\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") "
Aug 05 17:45:57.075774 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075576    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-model-cache\") pod \"ee62092a-fc94-47f6-bef3-8ac32b125367\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") "
Aug 05 17:45:57.075774 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075612    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-kserve-provision-location\") pod \"ee62092a-fc94-47f6-bef3-8ac32b125367\" (UID: \"ee62092a-fc94-47f6-bef3-8ac32b125367\") "
Aug 05 17:45:57.075774 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075637    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-home" (OuterVolumeSpecName: "home") pod "ee62092a-fc94-47f6-bef3-8ac32b125367" (UID: "ee62092a-fc94-47f6-bef3-8ac32b125367"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:45:57.075774 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075694    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ee62092a-fc94-47f6-bef3-8ac32b125367" (UID: "ee62092a-fc94-47f6-bef3-8ac32b125367"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:45:57.075977 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075822    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-model-cache" (OuterVolumeSpecName: "model-cache") pod "ee62092a-fc94-47f6-bef3-8ac32b125367" (UID: "ee62092a-fc94-47f6-bef3-8ac32b125367"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:45:57.075977 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075853    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:45:57.075977 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075871    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:45:57.075977 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.075925    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee62092a-fc94-47f6-bef3-8ac32b125367-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ee62092a-fc94-47f6-bef3-8ac32b125367" (UID: "ee62092a-fc94-47f6-bef3-8ac32b125367"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 17:45:57.077575 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.077545    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee62092a-fc94-47f6-bef3-8ac32b125367-kube-api-access-tv4zc" (OuterVolumeSpecName: "kube-api-access-tv4zc") pod "ee62092a-fc94-47f6-bef3-8ac32b125367" (UID: "ee62092a-fc94-47f6-bef3-8ac32b125367"). InnerVolumeSpecName "kube-api-access-tv4zc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:45:57.077681 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.077572    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee62092a-fc94-47f6-bef3-8ac32b125367-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ee62092a-fc94-47f6-bef3-8ac32b125367" (UID: "ee62092a-fc94-47f6-bef3-8ac32b125367"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:45:57.077911 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.077896    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-dshm" (OuterVolumeSpecName: "dshm") pod "ee62092a-fc94-47f6-bef3-8ac32b125367" (UID: "ee62092a-fc94-47f6-bef3-8ac32b125367"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:45:57.096562 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.096539    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ee62092a-fc94-47f6-bef3-8ac32b125367" (UID: "ee62092a-fc94-47f6-bef3-8ac32b125367"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:45:57.176622 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.176593    2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ee62092a-fc94-47f6-bef3-8ac32b125367-cabundle-cert\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:45:57.176622 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.176618    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:45:57.176769 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.176631    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-kserve-provision-location\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:45:57.176769 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.176644    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tv4zc\" (UniqueName: \"kubernetes.io/projected/ee62092a-fc94-47f6-bef3-8ac32b125367-kube-api-access-tv4zc\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:45:57.176769 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.176655    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee62092a-fc94-47f6-bef3-8ac32b125367-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:45:57.176769 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.176667    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee62092a-fc94-47f6-bef3-8ac32b125367-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:45:57.276403 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.276374    2574 generic.go:356] "Generic (PLEG): container finished" podID="ee62092a-fc94-47f6-bef3-8ac32b125367" containerID="724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1" exitCode=0
Aug 05 17:45:57.276522 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.276421    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"
Aug 05 17:45:57.276522 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.276456    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" event={"ID":"ee62092a-fc94-47f6-bef3-8ac32b125367","Type":"ContainerDied","Data":"724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1"}
Aug 05 17:45:57.276522 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.276507    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r" event={"ID":"ee62092a-fc94-47f6-bef3-8ac32b125367","Type":"ContainerDied","Data":"408c37449a0b7bb4aa4f08d5071fad4dc8892757f10d8880f9f54e96113a425a"}
Aug 05 17:45:57.276627 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.276529    2574 scope.go:117] "RemoveContainer" containerID="724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1"
Aug 05 17:45:57.285310 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.285295    2574 scope.go:117] "RemoveContainer" containerID="dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4"
Aug 05 17:45:57.292271 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.292255    2574 scope.go:117] "RemoveContainer" containerID="724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1"
Aug 05 17:45:57.292494 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:45:57.292480    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1\": container with ID starting with 724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1 not found: ID does not exist" containerID="724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1"
Aug 05 17:45:57.292538 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.292501    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1"} err="failed to get container status \"724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1\": rpc error: code = NotFound desc = could not find container \"724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1\": container with ID starting with 724b66618bf6e863c7c23f36a75e86bd25bd086aa3449872530af319e21b4fd1 not found: ID does not exist"
Aug 05 17:45:57.292538 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.292516    2574 scope.go:117] "RemoveContainer" containerID="dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4"
Aug 05 17:45:57.292729 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:45:57.292715    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4\": container with ID starting with dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4 not found: ID does not exist" containerID="dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4"
Aug 05 17:45:57.292772 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.292731    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4"} err="failed to get container status \"dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4\": rpc error: code = NotFound desc = could not find container \"dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4\": container with ID starting with dc257459583e8747a1de03a93aba2aeb3efd37c93a9636e0649fea0ef312f2a4 not found: ID does not exist"
Aug 05 17:45:57.300302 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.300282    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"]
Aug 05 17:45:57.301582 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.301561    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-0eb24714/tokenizer-template-args-test-kserve-6cf4bd5bc-nc78r"]
Aug 05 17:45:57.853514 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:45:57.853481    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee62092a-fc94-47f6-bef3-8ac32b125367" path="/var/lib/kubelet/pods/ee62092a-fc94-47f6-bef3-8ac32b125367/volumes"
Aug 05 17:52:45.916029 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.915990    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"]
Aug 05 17:52:45.916593 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.916399    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee62092a-fc94-47f6-bef3-8ac32b125367" containerName="storage-initializer"
Aug 05 17:52:45.916593 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.916419    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee62092a-fc94-47f6-bef3-8ac32b125367" containerName="storage-initializer"
Aug 05 17:52:45.916593 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.916434    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee62092a-fc94-47f6-bef3-8ac32b125367" containerName="main"
Aug 05 17:52:45.916593 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.916442    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee62092a-fc94-47f6-bef3-8ac32b125367" containerName="main"
Aug 05 17:52:45.916593 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.916512    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee62092a-fc94-47f6-bef3-8ac32b125367" containerName="main"
Aug 05 17:52:45.918357 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.918338    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:45.920623 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.920604    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"default-dockercfg-54kk4\""
Aug 05 17:52:45.921521 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.921506    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"kube-root-ca.crt\""
Aug 05 17:52:45.921597 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.921504    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"llmisv08544b88a8d963ffd553cc1f3ed82d16-kserve-self-signed-certs\""
Aug 05 17:52:45.921642 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.921513    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62fead0b\"/\"openshift-service-ca.crt\""
Aug 05 17:52:45.932140 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.932117    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"]
Aug 05 17:52:45.975927 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.975896    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx7dd\" (UniqueName: \"kubernetes.io/projected/fcef5f9d-ee93-4549-a1da-c1641906c3ca-kube-api-access-kx7dd\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:45.976077 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.975934    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:45.976077 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.975966    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:45.976077 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.976035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:45.976077 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.976076    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:45.976229 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:45.976102    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.076858 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.076830    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.076974 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.076871    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx7dd\" (UniqueName: \"kubernetes.io/projected/fcef5f9d-ee93-4549-a1da-c1641906c3ca-kube-api-access-kx7dd\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.076974 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.076905    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.076974 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.076935    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.077161 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.076975    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.077161 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.077127    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.077319 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.077298    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tmp-dir\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.077438 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.077369    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-home\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.077438 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.077430    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-model-cache\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.079179 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.079153    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-dshm\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.079421 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.079403    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tls-certs\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.085504 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.085465    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx7dd\" (UniqueName: \"kubernetes.io/projected/fcef5f9d-ee93-4549-a1da-c1641906c3ca-kube-api-access-kx7dd\") pod \"llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") " pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.228112 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.228046    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:46.359767 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.359700    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"]
Aug 05 17:52:46.362145 ip-10-0-141-246 kubenswrapper[2574]: W0805 17:52:46.362124    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcef5f9d_ee93_4549_a1da_c1641906c3ca.slice/crio-7de56bb87b1803ee48adc2db40ae95cdad0761ac630e988e97aec0e1f3732769 WatchSource:0}: Error finding container 7de56bb87b1803ee48adc2db40ae95cdad0761ac630e988e97aec0e1f3732769: Status 404 returned error can't find the container with id 7de56bb87b1803ee48adc2db40ae95cdad0761ac630e988e97aec0e1f3732769
Aug 05 17:52:46.799664 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.799625    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d" event={"ID":"fcef5f9d-ee93-4549-a1da-c1641906c3ca","Type":"ContainerStarted","Data":"8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c"}
Aug 05 17:52:46.799842 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.799722    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d" event={"ID":"fcef5f9d-ee93-4549-a1da-c1641906c3ca","Type":"ContainerStarted","Data":"7de56bb87b1803ee48adc2db40ae95cdad0761ac630e988e97aec0e1f3732769"}
Aug 05 17:52:46.817429 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:46.817377    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d" podStartSLOduration=1.817359893 podStartE2EDuration="1.817359893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-08-05 17:52:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 17:52:46.817042451 +0000 UTC m=+2411.516297794" watchObservedRunningTime="2026-08-05 17:52:46.817359893 +0000 UTC m=+2411.516615238"
Aug 05 17:52:47.228939 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:47.228908    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:56.228998 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:56.228964    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:56.241388 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:56.241364    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:52:56.846368 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:52:56.846340    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:53:19.840146 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:19.840113    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"]
Aug 05 17:53:19.850276 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:53:19.850239    2574 log.go:32] "ExecSync cmd from runtime service failed" err=<
Aug 05 17:53:19.850276 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:53:19.850276 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:53:19.850276 ip-10-0-141-246 kubenswrapper[2574]:  > containerID="8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c" cmd=["/bin/sleep","15"]
Aug 05 17:53:19.850607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:19.850292    2574 handlers.go:82] "Exec lifecycle hook for Container in Pod failed" execCommand=["/bin/sleep","15"] containerName="main" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d" message="" err=<
Aug 05 17:53:19.850607 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:53:19.850607 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:53:19.850607 ip-10-0-141-246 kubenswrapper[2574]:  >
Aug 05 17:53:19.850607 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:53:19.850304    2574 kuberuntime_container.go:755] "PreStop hook failed" err=<
Aug 05 17:53:19.850607 ip-10-0-141-246 kubenswrapper[2574]:         rpc error: code = Unknown desc = command error: executable file `/bin/sleep` not found: No such file or directory
Aug 05 17:53:19.850607 ip-10-0-141-246 kubenswrapper[2574]:         , stdout: , stderr: , exit code -1
Aug 05 17:53:19.850607 ip-10-0-141-246 kubenswrapper[2574]:  > pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d" podUID="fcef5f9d-ee93-4549-a1da-c1641906c3ca" containerName="main" containerID="cri-o://8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c"
Aug 05 17:53:19.850607 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:19.850347    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d" podUID="fcef5f9d-ee93-4549-a1da-c1641906c3ca" containerName="main" containerID="cri-o://8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c" gracePeriod=60
Aug 05 17:53:20.192509 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.192487    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:53:20.354690 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.354646    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx7dd\" (UniqueName: \"kubernetes.io/projected/fcef5f9d-ee93-4549-a1da-c1641906c3ca-kube-api-access-kx7dd\") pod \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") "
Aug 05 17:53:20.354856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.354693    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-home\") pod \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") "
Aug 05 17:53:20.354856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.354744    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tmp-dir\") pod \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") "
Aug 05 17:53:20.354856 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.354843    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-model-cache\") pod \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") "
Aug 05 17:53:20.354957 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.354914    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tls-certs\") pod \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") "
Aug 05 17:53:20.354999 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.354969    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-dshm\") pod \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\" (UID: \"fcef5f9d-ee93-4549-a1da-c1641906c3ca\") "
Aug 05 17:53:20.355086 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.354967    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fcef5f9d-ee93-4549-a1da-c1641906c3ca" (UID: "fcef5f9d-ee93-4549-a1da-c1641906c3ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:20.355086 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.355057    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-home" (OuterVolumeSpecName: "home") pod "fcef5f9d-ee93-4549-a1da-c1641906c3ca" (UID: "fcef5f9d-ee93-4549-a1da-c1641906c3ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:20.355201 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.355122    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-model-cache" (OuterVolumeSpecName: "model-cache") pod "fcef5f9d-ee93-4549-a1da-c1641906c3ca" (UID: "fcef5f9d-ee93-4549-a1da-c1641906c3ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:20.355249 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.355229    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:53:20.355305 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.355249    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:53:20.355305 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.355262    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:53:20.356930 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.356909    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-dshm" (OuterVolumeSpecName: "dshm") pod "fcef5f9d-ee93-4549-a1da-c1641906c3ca" (UID: "fcef5f9d-ee93-4549-a1da-c1641906c3ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 17:53:20.357035 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.357003    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcef5f9d-ee93-4549-a1da-c1641906c3ca-kube-api-access-kx7dd" (OuterVolumeSpecName: "kube-api-access-kx7dd") pod "fcef5f9d-ee93-4549-a1da-c1641906c3ca" (UID: "fcef5f9d-ee93-4549-a1da-c1641906c3ca"). InnerVolumeSpecName "kube-api-access-kx7dd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 17:53:20.357084 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.357069    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fcef5f9d-ee93-4549-a1da-c1641906c3ca" (UID: "fcef5f9d-ee93-4549-a1da-c1641906c3ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 17:53:20.456104 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.456067    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcef5f9d-ee93-4549-a1da-c1641906c3ca-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:53:20.456104 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.456100    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcef5f9d-ee93-4549-a1da-c1641906c3ca-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:53:20.456255 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.456114    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx7dd\" (UniqueName: \"kubernetes.io/projected/fcef5f9d-ee93-4549-a1da-c1641906c3ca-kube-api-access-kx7dd\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 17:53:20.923775 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.923738    2574 generic.go:356] "Generic (PLEG): container finished" podID="fcef5f9d-ee93-4549-a1da-c1641906c3ca" containerID="8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c" exitCode=0
Aug 05 17:53:20.923775 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.923775    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d" event={"ID":"fcef5f9d-ee93-4549-a1da-c1641906c3ca","Type":"ContainerDied","Data":"8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c"}
Aug 05 17:53:20.924366 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.923792    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"
Aug 05 17:53:20.924366 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.923812    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d" event={"ID":"fcef5f9d-ee93-4549-a1da-c1641906c3ca","Type":"ContainerDied","Data":"7de56bb87b1803ee48adc2db40ae95cdad0761ac630e988e97aec0e1f3732769"}
Aug 05 17:53:20.924366 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.923832    2574 scope.go:117] "RemoveContainer" containerID="8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c"
Aug 05 17:53:20.932372 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.932355    2574 scope.go:117] "RemoveContainer" containerID="8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c"
Aug 05 17:53:20.932610 ip-10-0-141-246 kubenswrapper[2574]: E0805 17:53:20.932590    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c\": container with ID starting with 8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c not found: ID does not exist" containerID="8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c"
Aug 05 17:53:20.932646 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.932620    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c"} err="failed to get container status \"8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c\": rpc error: code = NotFound desc = could not find container \"8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c\": container with ID starting with 8f7028cea836b3f9a0fb846c66db244737ef74837186887704de9b0a62cc5c5c not found: ID does not exist"
Aug 05 17:53:20.944420 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.944395    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"]
Aug 05 17:53:20.949743 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:20.949724    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62fead0b/llmisvc-model-qwen2-5-0-5b-rout-4f8c0978-kserve-bb749b474-rgl4d"]
Aug 05 17:53:21.854404 ip-10-0-141-246 kubenswrapper[2574]: I0805 17:53:21.854362    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcef5f9d-ee93-4549-a1da-c1641906c3ca" path="/var/lib/kubelet/pods/fcef5f9d-ee93-4549-a1da-c1641906c3ca/volumes"
Aug 05 18:01:20.615846 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.615807    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"]
Aug 05 18:01:20.618180 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.616093    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcef5f9d-ee93-4549-a1da-c1641906c3ca" containerName="main"
Aug 05 18:01:20.618180 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.616105    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcef5f9d-ee93-4549-a1da-c1641906c3ca" containerName="main"
Aug 05 18:01:20.618180 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.616168    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcef5f9d-ee93-4549-a1da-c1641906c3ca" containerName="main"
Aug 05 18:01:20.619135 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.619103    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.621251 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.621230    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"seaweedfs-s3-creds\""
Aug 05 18:01:20.621251 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.621243    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"odh-kserve-custom-ca-bundle\""
Aug 05 18:01:20.621424 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.621408    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"openshift-service-ca.crt\""
Aug 05 18:01:20.621596 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.621567    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"llmisva690bbc929faec8bc98c767f16c003c1-kserve-self-signed-certs\""
Aug 05 18:01:20.621718 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.621609    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"kube-root-ca.crt\""
Aug 05 18:01:20.621887 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.621871    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-079cb970\"/\"default-dockercfg-64xwq\""
Aug 05 18:01:20.629706 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.629683    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"]
Aug 05 18:01:20.760980 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.760944    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.761196 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.760985    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-dshm\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.761196 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.761039    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-home\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.761196 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.761064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.761196 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.761123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.761196 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.761178    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.761455 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.761202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkm6k\" (UniqueName: \"kubernetes.io/projected/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kube-api-access-wkm6k\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.761455 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.761255    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.862328 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.862293    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.862328 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.862334    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-dshm\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.862584 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.862362    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-home\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.862584 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.862466    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.862584 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.862531    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.862584 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.862567    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.862584 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.862585    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkm6k\" (UniqueName: \"kubernetes.io/projected/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kube-api-access-wkm6k\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.862830 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.862620    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.862830 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.862735    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-home\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.863252 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.863226    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.863353 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.863233    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.863353 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.863233    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.863353 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.863319    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.864688 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.864657    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-dshm\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.864898 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.864882    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.869890 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.869849    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkm6k\" (UniqueName: \"kubernetes.io/projected/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kube-api-access-wkm6k\") pod \"llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") " pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:20.929733 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:20.929683    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:21.063984 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:21.063955    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"]
Aug 05 18:01:21.066121 ip-10-0-141-246 kubenswrapper[2574]: W0805 18:01:21.066081    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod045f571c_6e8a_4d43_85be_b0dcd48e7ea8.slice/crio-507400737c5a9ec40507df0f553282e9e6acb0a148ac00bc5fd609b3d1f4770a WatchSource:0}: Error finding container 507400737c5a9ec40507df0f553282e9e6acb0a148ac00bc5fd609b3d1f4770a: Status 404 returned error can't find the container with id 507400737c5a9ec40507df0f553282e9e6acb0a148ac00bc5fd609b3d1f4770a
Aug 05 18:01:21.678309 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:21.678267    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" event={"ID":"045f571c-6e8a-4d43-85be-b0dcd48e7ea8","Type":"ContainerStarted","Data":"09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d"}
Aug 05 18:01:21.678309 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:21.678317    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" event={"ID":"045f571c-6e8a-4d43-85be-b0dcd48e7ea8","Type":"ContainerStarted","Data":"507400737c5a9ec40507df0f553282e9e6acb0a148ac00bc5fd609b3d1f4770a"}
Aug 05 18:01:23.689439 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:23.689406    2574 generic.go:356] "Generic (PLEG): container finished" podID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerID="09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d" exitCode=0
Aug 05 18:01:23.689861 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:23.689477    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" event={"ID":"045f571c-6e8a-4d43-85be-b0dcd48e7ea8","Type":"ContainerDied","Data":"09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d"}
Aug 05 18:01:24.695465 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:24.695417    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" event={"ID":"045f571c-6e8a-4d43-85be-b0dcd48e7ea8","Type":"ContainerStarted","Data":"17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8"}
Aug 05 18:01:24.717283 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:24.717216    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podStartSLOduration=2.600194836 podStartE2EDuration="4.717194477s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.116999641s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:01:24.714732727 +0000 UTC m=+2929.413988082" watchObservedRunningTime="2026-08-05 18:01:24.717194477 +0000 UTC m=+2929.416449820"
Aug 05 18:01:24.930574 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:24.930531    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:30.930917 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:30.930877    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:01:30.932519 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:30.932488    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:01:40.930233 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:40.930178    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:01:50.930549 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:01:50.930507    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:02:00.930374 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:02:00.930275    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:02:10.930472 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:02:10.930430    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:02:20.931031 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:02:20.930969    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:02:30.930619 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:02:30.930579    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:02:40.930880 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:02:40.930836    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:02:50.930458 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:02:50.930410    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:03:00.930672 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:00.930630    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:03:10.930987 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:10.930938    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": dial tcp 10.134.0.29:8000: connect: connection refused"
Aug 05 18:03:20.940697 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:20.940668    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:03:20.948318 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:20.948284    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:03:25.314636 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:25.314603    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"]
Aug 05 18:03:40.336401 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:40.336297    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" containerID="cri-o://17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8" gracePeriod=45
Aug 05 18:03:40.975182 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:40.975155    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:03:41.003412 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.003380    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-cabundle-cert\") pod \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") "
Aug 05 18:03:41.003412 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.003413    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkm6k\" (UniqueName: \"kubernetes.io/projected/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kube-api-access-wkm6k\") pod \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") "
Aug 05 18:03:41.003604 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.003436    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-dshm\") pod \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") "
Aug 05 18:03:41.003604 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.003460    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kserve-provision-location\") pod \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") "
Aug 05 18:03:41.003716 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.003614    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-home\") pod \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") "
Aug 05 18:03:41.003716 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.003660    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tls-certs\") pod \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") "
Aug 05 18:03:41.003824 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.003722    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-model-cache\") pod \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") "
Aug 05 18:03:41.003824 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.003746    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tmp-dir\") pod \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\" (UID: \"045f571c-6e8a-4d43-85be-b0dcd48e7ea8\") "
Aug 05 18:03:41.003926 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.003788    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "045f571c-6e8a-4d43-85be-b0dcd48e7ea8" (UID: "045f571c-6e8a-4d43-85be-b0dcd48e7ea8"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:03:41.004086 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.004066    2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-cabundle-cert\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:03:41.004167 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.004077    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-model-cache" (OuterVolumeSpecName: "model-cache") pod "045f571c-6e8a-4d43-85be-b0dcd48e7ea8" (UID: "045f571c-6e8a-4d43-85be-b0dcd48e7ea8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:03:41.004290 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.004269    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-home" (OuterVolumeSpecName: "home") pod "045f571c-6e8a-4d43-85be-b0dcd48e7ea8" (UID: "045f571c-6e8a-4d43-85be-b0dcd48e7ea8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:03:41.006241 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.006194    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "045f571c-6e8a-4d43-85be-b0dcd48e7ea8" (UID: "045f571c-6e8a-4d43-85be-b0dcd48e7ea8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:03:41.006326 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.006256    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kube-api-access-wkm6k" (OuterVolumeSpecName: "kube-api-access-wkm6k") pod "045f571c-6e8a-4d43-85be-b0dcd48e7ea8" (UID: "045f571c-6e8a-4d43-85be-b0dcd48e7ea8"). InnerVolumeSpecName "kube-api-access-wkm6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:03:41.006370 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.006322    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-dshm" (OuterVolumeSpecName: "dshm") pod "045f571c-6e8a-4d43-85be-b0dcd48e7ea8" (UID: "045f571c-6e8a-4d43-85be-b0dcd48e7ea8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:03:41.017111 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.017090    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "045f571c-6e8a-4d43-85be-b0dcd48e7ea8" (UID: "045f571c-6e8a-4d43-85be-b0dcd48e7ea8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:03:41.025393 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.025373    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "045f571c-6e8a-4d43-85be-b0dcd48e7ea8" (UID: "045f571c-6e8a-4d43-85be-b0dcd48e7ea8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:03:41.104442 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.104386    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:03:41.104442 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.104410    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:03:41.104442 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.104420    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkm6k\" (UniqueName: \"kubernetes.io/projected/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kube-api-access-wkm6k\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:03:41.104442 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.104428    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:03:41.104442 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.104436    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-kserve-provision-location\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:03:41.104442 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.104444    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:03:41.104669 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.104452    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/045f571c-6e8a-4d43-85be-b0dcd48e7ea8-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:03:41.218783 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.218752    2574 generic.go:356] "Generic (PLEG): container finished" podID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerID="17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8" exitCode=0
Aug 05 18:03:41.218870 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.218811    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"
Aug 05 18:03:41.218921 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.218810    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" event={"ID":"045f571c-6e8a-4d43-85be-b0dcd48e7ea8","Type":"ContainerDied","Data":"17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8"}
Aug 05 18:03:41.218963 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.218935    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" event={"ID":"045f571c-6e8a-4d43-85be-b0dcd48e7ea8","Type":"ContainerDied","Data":"507400737c5a9ec40507df0f553282e9e6acb0a148ac00bc5fd609b3d1f4770a"}
Aug 05 18:03:41.218999 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.218965    2574 scope.go:117] "RemoveContainer" containerID="17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8"
Aug 05 18:03:41.227887 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.227871    2574 scope.go:117] "RemoveContainer" containerID="09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d"
Aug 05 18:03:41.235149 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.235085    2574 scope.go:117] "RemoveContainer" containerID="17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8"
Aug 05 18:03:41.235349 ip-10-0-141-246 kubenswrapper[2574]: E0805 18:03:41.235330    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8\": container with ID starting with 17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8 not found: ID does not exist" containerID="17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8"
Aug 05 18:03:41.235396 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.235357    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8"} err="failed to get container status \"17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8\": rpc error: code = NotFound desc = could not find container \"17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8\": container with ID starting with 17bcbfcd08c54120bf490877cc3f0eee7628cbdf9b154a29691d6ff76281bdc8 not found: ID does not exist"
Aug 05 18:03:41.235396 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.235373    2574 scope.go:117] "RemoveContainer" containerID="09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d"
Aug 05 18:03:41.235610 ip-10-0-141-246 kubenswrapper[2574]: E0805 18:03:41.235592    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d\": container with ID starting with 09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d not found: ID does not exist" containerID="09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d"
Aug 05 18:03:41.235664 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.235619    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d"} err="failed to get container status \"09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d\": rpc error: code = NotFound desc = could not find container \"09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d\": container with ID starting with 09c96ad26769fa037bfdcf6c61086ae4cf22247e243a535a5c7dadd5f287e67d not found: ID does not exist"
Aug 05 18:03:41.240252 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.240231    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"]
Aug 05 18:03:41.243414 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.243397    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd"]
Aug 05 18:03:41.854394 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.854365    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" path="/var/lib/kubelet/pods/045f571c-6e8a-4d43-85be-b0dcd48e7ea8/volumes"
Aug 05 18:03:41.942300 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:03:41.942269    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-test-llm-inference-service-079cb970/llmisvc-model-fb-opt-125m-route-87882a8e-kserve-567855d745rk4sd" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main" probeResult="failure" output="Get \"https://10.134.0.29:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Aug 05 18:09:23.102072 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.102041    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"]
Aug 05 18:09:23.102588 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.102338    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main"
Aug 05 18:09:23.102588 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.102351    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main"
Aug 05 18:09:23.102588 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.102359    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="storage-initializer"
Aug 05 18:09:23.102588 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.102365    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="storage-initializer"
Aug 05 18:09:23.102588 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.102406    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="045f571c-6e8a-4d43-85be-b0dcd48e7ea8" containerName="main"
Aug 05 18:09:23.105595 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.105576    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.107909 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.107873    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"llmisv44d181485fad85e662eb092f3749502f-kserve-self-signed-certs\""
Aug 05 18:09:23.107909 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.107886    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"openshift-service-ca.crt\""
Aug 05 18:09:23.108135 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.107886    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"odh-kserve-custom-ca-bundle\""
Aug 05 18:09:23.108135 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.107907    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"kube-root-ca.crt\""
Aug 05 18:09:23.108135 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.108026    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"default-dockercfg-mwxt8\""
Aug 05 18:09:23.108346 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.108326    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-62876e81\"/\"seaweedfs-s3-creds\""
Aug 05 18:09:23.116076 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.116055    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"]
Aug 05 18:09:23.186183 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.186155    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4c8edf-99e8-4191-b066-7e4d572038ed-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.186293 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.186187    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-dshm\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.186293 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.186205    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de4c8edf-99e8-4191-b066-7e4d572038ed-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.186293 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.186232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.186293 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.186260    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-home\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.186293 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.186282    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.186456 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.186299    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.186456 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.186338    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqdxf\" (UniqueName: \"kubernetes.io/projected/de4c8edf-99e8-4191-b066-7e4d572038ed-kube-api-access-tqdxf\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287181 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287341 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287184    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287341 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287215    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqdxf\" (UniqueName: \"kubernetes.io/projected/de4c8edf-99e8-4191-b066-7e4d572038ed-kube-api-access-tqdxf\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287542 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287360    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4c8edf-99e8-4191-b066-7e4d572038ed-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287542 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-dshm\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287542 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de4c8edf-99e8-4191-b066-7e4d572038ed-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287542 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287486    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287542 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287532    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-home\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287799 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287559    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-model-cache\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287799 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287670    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-kserve-provision-location\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287907 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287798    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-tmp-dir\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.287907 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.287826    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-home\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.288222 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.288203    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de4c8edf-99e8-4191-b066-7e4d572038ed-cabundle-cert\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.289534 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.289515    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-dshm\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.289868 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.289851    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4c8edf-99e8-4191-b066-7e4d572038ed-tls-certs\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.294546 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.294524    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqdxf\" (UniqueName: \"kubernetes.io/projected/de4c8edf-99e8-4191-b066-7e4d572038ed-kube-api-access-tqdxf\") pod \"llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") " pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.416611 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.416583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:23.545623 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:23.545499    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"]
Aug 05 18:09:23.548046 ip-10-0-141-246 kubenswrapper[2574]: W0805 18:09:23.548005    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4c8edf_99e8_4191_b066_7e4d572038ed.slice/crio-4752b0e716311f1ac1eeca5e1ee9954144b415ddf067e628fd38c8d9d3ca717a WatchSource:0}: Error finding container 4752b0e716311f1ac1eeca5e1ee9954144b415ddf067e628fd38c8d9d3ca717a: Status 404 returned error can't find the container with id 4752b0e716311f1ac1eeca5e1ee9954144b415ddf067e628fd38c8d9d3ca717a
Aug 05 18:09:24.478082 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:24.478039    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" event={"ID":"de4c8edf-99e8-4191-b066-7e4d572038ed","Type":"ContainerStarted","Data":"c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f"}
Aug 05 18:09:24.478579 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:24.478108    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" event={"ID":"de4c8edf-99e8-4191-b066-7e4d572038ed","Type":"ContainerStarted","Data":"4752b0e716311f1ac1eeca5e1ee9954144b415ddf067e628fd38c8d9d3ca717a"}
Aug 05 18:09:26.486805 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:26.486770    2574 generic.go:356] "Generic (PLEG): container finished" podID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerID="c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f" exitCode=0
Aug 05 18:09:26.487199 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:26.486844    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" event={"ID":"de4c8edf-99e8-4191-b066-7e4d572038ed","Type":"ContainerDied","Data":"c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f"}
Aug 05 18:09:27.492680 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:27.492647    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" event={"ID":"de4c8edf-99e8-4191-b066-7e4d572038ed","Type":"ContainerStarted","Data":"406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28"}
Aug 05 18:09:27.516496 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:27.516443    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podStartSLOduration=2.501730625 podStartE2EDuration="4.516426238s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.014695613s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:09:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:09:27.515196473 +0000 UTC m=+3412.214451815" watchObservedRunningTime="2026-08-05 18:09:27.516426238 +0000 UTC m=+3412.215681580"
Aug 05 18:09:33.417178 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:33.417138    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:33.417634 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:33.417237    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:09:33.418678 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:33.418650    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:09:43.417524 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:43.417479    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:09:53.417249 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:09:53.417202    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:10:03.417675 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:10:03.417629    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:10:13.417693 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:10:13.417647    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:10:23.417967 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:10:23.417919    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:10:33.417040 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:10:33.416974    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:10:43.417259 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:10:43.417210    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:10:53.417361 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:10:53.417319    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:11:03.417902 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:03.417808    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" probeResult="failure" output="Get \"https://10.134.0.30:8000/health\": dial tcp 10.134.0.30:8000: connect: connection refused"
Aug 05 18:11:13.426640 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:13.426611    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:11:13.434541 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:13.434518    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:11:35.469122 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:35.469088    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"]
Aug 05 18:11:50.493705 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:50.493645    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main" containerID="cri-o://406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28" gracePeriod=45
Aug 05 18:11:51.234283 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.234257    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:11:51.301602 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.301535    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-dshm\") pod \"de4c8edf-99e8-4191-b066-7e4d572038ed\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") "
Aug 05 18:11:51.301602 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.301570    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4c8edf-99e8-4191-b066-7e4d572038ed-tls-certs\") pod \"de4c8edf-99e8-4191-b066-7e4d572038ed\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") "
Aug 05 18:11:51.301602 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.301592    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-home\") pod \"de4c8edf-99e8-4191-b066-7e4d572038ed\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") "
Aug 05 18:11:51.301848 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.301652    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de4c8edf-99e8-4191-b066-7e4d572038ed-cabundle-cert\") pod \"de4c8edf-99e8-4191-b066-7e4d572038ed\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") "
Aug 05 18:11:51.301848 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.301683    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-model-cache\") pod \"de4c8edf-99e8-4191-b066-7e4d572038ed\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") "
Aug 05 18:11:51.301848 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.301714    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-tmp-dir\") pod \"de4c8edf-99e8-4191-b066-7e4d572038ed\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") "
Aug 05 18:11:51.301848 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.301741    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqdxf\" (UniqueName: \"kubernetes.io/projected/de4c8edf-99e8-4191-b066-7e4d572038ed-kube-api-access-tqdxf\") pod \"de4c8edf-99e8-4191-b066-7e4d572038ed\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") "
Aug 05 18:11:51.301848 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.301786    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-kserve-provision-location\") pod \"de4c8edf-99e8-4191-b066-7e4d572038ed\" (UID: \"de4c8edf-99e8-4191-b066-7e4d572038ed\") "
Aug 05 18:11:51.302137 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.301948    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-model-cache" (OuterVolumeSpecName: "model-cache") pod "de4c8edf-99e8-4191-b066-7e4d572038ed" (UID: "de4c8edf-99e8-4191-b066-7e4d572038ed"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:11:51.302137 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.302051    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de4c8edf-99e8-4191-b066-7e4d572038ed-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "de4c8edf-99e8-4191-b066-7e4d572038ed" (UID: "de4c8edf-99e8-4191-b066-7e4d572038ed"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:11:51.302137 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.302067    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:11:51.302492 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.302463    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-home" (OuterVolumeSpecName: "home") pod "de4c8edf-99e8-4191-b066-7e4d572038ed" (UID: "de4c8edf-99e8-4191-b066-7e4d572038ed"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:11:51.303895 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.303871    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de4c8edf-99e8-4191-b066-7e4d572038ed-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de4c8edf-99e8-4191-b066-7e4d572038ed" (UID: "de4c8edf-99e8-4191-b066-7e4d572038ed"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:11:51.304067 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.304050    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-dshm" (OuterVolumeSpecName: "dshm") pod "de4c8edf-99e8-4191-b066-7e4d572038ed" (UID: "de4c8edf-99e8-4191-b066-7e4d572038ed"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:11:51.304242 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.304224    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de4c8edf-99e8-4191-b066-7e4d572038ed-kube-api-access-tqdxf" (OuterVolumeSpecName: "kube-api-access-tqdxf") pod "de4c8edf-99e8-4191-b066-7e4d572038ed" (UID: "de4c8edf-99e8-4191-b066-7e4d572038ed"). InnerVolumeSpecName "kube-api-access-tqdxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:11:51.315357 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.315313    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de4c8edf-99e8-4191-b066-7e4d572038ed" (UID: "de4c8edf-99e8-4191-b066-7e4d572038ed"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:11:51.323216 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.323196    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "de4c8edf-99e8-4191-b066-7e4d572038ed" (UID: "de4c8edf-99e8-4191-b066-7e4d572038ed"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:11:51.402916 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.402891    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:11:51.402916 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.402913    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de4c8edf-99e8-4191-b066-7e4d572038ed-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:11:51.403103 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.402928    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:11:51.403103 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.402941    2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/de4c8edf-99e8-4191-b066-7e4d572038ed-cabundle-cert\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:11:51.403103 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.402951    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:11:51.403103 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.402963    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqdxf\" (UniqueName: \"kubernetes.io/projected/de4c8edf-99e8-4191-b066-7e4d572038ed-kube-api-access-tqdxf\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:11:51.403103 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:51.402975    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/de4c8edf-99e8-4191-b066-7e4d572038ed-kserve-provision-location\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:11:52.061359 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.061324    2574 generic.go:356] "Generic (PLEG): container finished" podID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerID="406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28" exitCode=0
Aug 05 18:11:52.061815 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.061385    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"
Aug 05 18:11:52.061815 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.061418    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" event={"ID":"de4c8edf-99e8-4191-b066-7e4d572038ed","Type":"ContainerDied","Data":"406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28"}
Aug 05 18:11:52.061815 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.061520    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl" event={"ID":"de4c8edf-99e8-4191-b066-7e4d572038ed","Type":"ContainerDied","Data":"4752b0e716311f1ac1eeca5e1ee9954144b415ddf067e628fd38c8d9d3ca717a"}
Aug 05 18:11:52.061815 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.061553    2574 scope.go:117] "RemoveContainer" containerID="406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28"
Aug 05 18:11:52.069718 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.069703    2574 scope.go:117] "RemoveContainer" containerID="c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f"
Aug 05 18:11:52.076681 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.076615    2574 scope.go:117] "RemoveContainer" containerID="406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28"
Aug 05 18:11:52.077430 ip-10-0-141-246 kubenswrapper[2574]: E0805 18:11:52.077401    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28\": container with ID starting with 406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28 not found: ID does not exist" containerID="406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28"
Aug 05 18:11:52.077519 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.077440    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28"} err="failed to get container status \"406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28\": rpc error: code = NotFound desc = could not find container \"406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28\": container with ID starting with 406aaac5683dd6903b0fac5c583140273b62404f1084386aa7a055ff4d6a4f28 not found: ID does not exist"
Aug 05 18:11:52.077519 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.077465    2574 scope.go:117] "RemoveContainer" containerID="c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f"
Aug 05 18:11:52.077739 ip-10-0-141-246 kubenswrapper[2574]: E0805 18:11:52.077718    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f\": container with ID starting with c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f not found: ID does not exist" containerID="c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f"
Aug 05 18:11:52.077822 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.077744    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f"} err="failed to get container status \"c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f\": rpc error: code = NotFound desc = could not find container \"c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f\": container with ID starting with c75902babfa30320a9d76b49632fd81d811b1c1987ca3d092f5ca118950d862f not found: ID does not exist"
Aug 05 18:11:52.078772 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.078753    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"]
Aug 05 18:11:52.082432 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:52.082413    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-62876e81/llmisvc-model-fb-opt-125m-route-50bc673d-kserve-prefill-5bnlpzl"]
Aug 05 18:11:53.854339 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:11:53.854307    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" path="/var/lib/kubelet/pods/de4c8edf-99e8-4191-b066-7e4d572038ed/volumes"
Aug 05 18:12:09.380389 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.380359    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"]
Aug 05 18:12:09.380840 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.380619    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main"
Aug 05 18:12:09.380840 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.380629    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main"
Aug 05 18:12:09.380840 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.380641    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="storage-initializer"
Aug 05 18:12:09.380840 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.380648    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="storage-initializer"
Aug 05 18:12:09.380840 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.380706    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de4c8edf-99e8-4191-b066-7e4d572038ed" containerName="main"
Aug 05 18:12:09.385208 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.385185    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.387392 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.387366    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"custom-route-timeout-pd-test-kserve-self-signed-certs\""
Aug 05 18:12:09.387528 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.387366    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"odh-kserve-custom-ca-bundle\""
Aug 05 18:12:09.387528 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.387499    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"kube-root-ca.crt\""
Aug 05 18:12:09.387632 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.387617    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"seaweedfs-s3-creds\""
Aug 05 18:12:09.387691 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.387677    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"default-dockercfg-ppb4w\""
Aug 05 18:12:09.388136 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.388120    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-d73d44f4\"/\"openshift-service-ca.crt\""
Aug 05 18:12:09.395681 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.395661    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"]
Aug 05 18:12:09.439997 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.439971    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kth6s\" (UniqueName: \"kubernetes.io/projected/07be1ce0-5500-459d-8a1f-f92422a290e7-kube-api-access-kth6s\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.440163 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.440035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.440163 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.440063    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07be1ce0-5500-459d-8a1f-f92422a290e7-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.440163 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.440079    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07be1ce0-5500-459d-8a1f-f92422a290e7-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.440163 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.440099    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.440163 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.440130    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.440346 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.440176    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.440346 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.440259    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.540657 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.540622    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kth6s\" (UniqueName: \"kubernetes.io/projected/07be1ce0-5500-459d-8a1f-f92422a290e7-kube-api-access-kth6s\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.540781 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.540665    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.540781 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.540690    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07be1ce0-5500-459d-8a1f-f92422a290e7-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.540781 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.540713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07be1ce0-5500-459d-8a1f-f92422a290e7-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.540781 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.540738    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.540781 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.540770    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.541003 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.540793    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.541003 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.540916    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.541281 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.541245    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-model-cache\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.541400 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.541318    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-tmp-dir\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.541400 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.541376    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-kserve-provision-location\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.541523 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.541435    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-home\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.541523 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.541487    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07be1ce0-5500-459d-8a1f-f92422a290e7-cabundle-cert\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.542943 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.542921    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-dshm\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.543316 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.543299    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07be1ce0-5500-459d-8a1f-f92422a290e7-tls-certs\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.551483 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.551461    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kth6s\" (UniqueName: \"kubernetes.io/projected/07be1ce0-5500-459d-8a1f-f92422a290e7-kube-api-access-kth6s\") pod \"custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") " pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.695451 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.695425    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:09.826478 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:09.826456    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"]
Aug 05 18:12:09.828965 ip-10-0-141-246 kubenswrapper[2574]: W0805 18:12:09.828936    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07be1ce0_5500_459d_8a1f_f92422a290e7.slice/crio-015dadd16262ac05a21b8df1d226eeeff35814a74d3c7e12fc04ff1b925fceae WatchSource:0}: Error finding container 015dadd16262ac05a21b8df1d226eeeff35814a74d3c7e12fc04ff1b925fceae: Status 404 returned error can't find the container with id 015dadd16262ac05a21b8df1d226eeeff35814a74d3c7e12fc04ff1b925fceae
Aug 05 18:12:10.129878 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:10.129793    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" event={"ID":"07be1ce0-5500-459d-8a1f-f92422a290e7","Type":"ContainerStarted","Data":"bcbce8540fad269fb803ac91b10437baa89de942d397f210ac9d16027a929ccc"}
Aug 05 18:12:10.129878 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:10.129849    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" event={"ID":"07be1ce0-5500-459d-8a1f-f92422a290e7","Type":"ContainerStarted","Data":"015dadd16262ac05a21b8df1d226eeeff35814a74d3c7e12fc04ff1b925fceae"}
Aug 05 18:12:12.139170 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:12.139138    2574 generic.go:356] "Generic (PLEG): container finished" podID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerID="bcbce8540fad269fb803ac91b10437baa89de942d397f210ac9d16027a929ccc" exitCode=0
Aug 05 18:12:12.139543 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:12.139191    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" event={"ID":"07be1ce0-5500-459d-8a1f-f92422a290e7","Type":"ContainerDied","Data":"bcbce8540fad269fb803ac91b10437baa89de942d397f210ac9d16027a929ccc"}
Aug 05 18:12:13.145679 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:13.145641    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" event={"ID":"07be1ce0-5500-459d-8a1f-f92422a290e7","Type":"ContainerStarted","Data":"f3b4dadf6ccfa63f368869415e3365c6eb3ca596859f5c5060fc91dbaa524cdd"}
Aug 05 18:12:13.167426 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:13.167368    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podStartSLOduration=2.151225246 podStartE2EDuration="4.167350327s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.016125081s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:12:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:12:13.166290276 +0000 UTC m=+3577.865545629" watchObservedRunningTime="2026-08-05 18:12:13.167350327 +0000 UTC m=+3577.866605672"
Aug 05 18:12:19.696674 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:19.696634    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:19.697115 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:19.696707    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:12:19.698239 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:19.698210    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:12:29.696228 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:29.696129    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:12:39.696066 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:39.695995    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:12:49.696143 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:49.696090    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:12:59.696185 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:12:59.696136    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:13:09.695919 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:13:09.695876    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:13:19.696850 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:13:19.696799    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:13:29.695929 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:13:29.695881    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:13:39.696080 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:13:39.696030    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:13:49.695826 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:13:49.695782    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:13:59.696759 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:13:59.696664    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" probeResult="failure" output="Get \"https://10.134.0.31:8000/health\": dial tcp 10.134.0.31:8000: connect: connection refused"
Aug 05 18:14:09.705674 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:09.705646    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:14:09.713912 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:09.713884    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:14:19.066199 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:19.066135    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"]
Aug 05 18:14:34.088467 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.088418    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main" containerID="cri-o://f3b4dadf6ccfa63f368869415e3365c6eb3ca596859f5c5060fc91dbaa524cdd" gracePeriod=45
Aug 05 18:14:34.709434 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.709406    2574 generic.go:356] "Generic (PLEG): container finished" podID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerID="f3b4dadf6ccfa63f368869415e3365c6eb3ca596859f5c5060fc91dbaa524cdd" exitCode=0
Aug 05 18:14:34.709594 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.709478    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" event={"ID":"07be1ce0-5500-459d-8a1f-f92422a290e7","Type":"ContainerDied","Data":"f3b4dadf6ccfa63f368869415e3365c6eb3ca596859f5c5060fc91dbaa524cdd"}
Aug 05 18:14:34.750702 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.750679    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:14:34.923375 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923352    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-model-cache\") pod \"07be1ce0-5500-459d-8a1f-f92422a290e7\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") "
Aug 05 18:14:34.923518 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923388    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-home\") pod \"07be1ce0-5500-459d-8a1f-f92422a290e7\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") "
Aug 05 18:14:34.923518 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923414    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-kserve-provision-location\") pod \"07be1ce0-5500-459d-8a1f-f92422a290e7\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") "
Aug 05 18:14:34.923518 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923450    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-tmp-dir\") pod \"07be1ce0-5500-459d-8a1f-f92422a290e7\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") "
Aug 05 18:14:34.923518 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923481    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kth6s\" (UniqueName: \"kubernetes.io/projected/07be1ce0-5500-459d-8a1f-f92422a290e7-kube-api-access-kth6s\") pod \"07be1ce0-5500-459d-8a1f-f92422a290e7\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") "
Aug 05 18:14:34.923518 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923509    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07be1ce0-5500-459d-8a1f-f92422a290e7-cabundle-cert\") pod \"07be1ce0-5500-459d-8a1f-f92422a290e7\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") "
Aug 05 18:14:34.923789 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923560    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-dshm\") pod \"07be1ce0-5500-459d-8a1f-f92422a290e7\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") "
Aug 05 18:14:34.923789 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923608    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07be1ce0-5500-459d-8a1f-f92422a290e7-tls-certs\") pod \"07be1ce0-5500-459d-8a1f-f92422a290e7\" (UID: \"07be1ce0-5500-459d-8a1f-f92422a290e7\") "
Aug 05 18:14:34.923789 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923654    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-model-cache" (OuterVolumeSpecName: "model-cache") pod "07be1ce0-5500-459d-8a1f-f92422a290e7" (UID: "07be1ce0-5500-459d-8a1f-f92422a290e7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:14:34.923789 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923783    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:14:34.924307 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.923970    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07be1ce0-5500-459d-8a1f-f92422a290e7-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "07be1ce0-5500-459d-8a1f-f92422a290e7" (UID: "07be1ce0-5500-459d-8a1f-f92422a290e7"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:14:34.924435 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.924407    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-home" (OuterVolumeSpecName: "home") pod "07be1ce0-5500-459d-8a1f-f92422a290e7" (UID: "07be1ce0-5500-459d-8a1f-f92422a290e7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:14:34.926880 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.926846    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-dshm" (OuterVolumeSpecName: "dshm") pod "07be1ce0-5500-459d-8a1f-f92422a290e7" (UID: "07be1ce0-5500-459d-8a1f-f92422a290e7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:14:34.926880 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.926856    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07be1ce0-5500-459d-8a1f-f92422a290e7-kube-api-access-kth6s" (OuterVolumeSpecName: "kube-api-access-kth6s") pod "07be1ce0-5500-459d-8a1f-f92422a290e7" (UID: "07be1ce0-5500-459d-8a1f-f92422a290e7"). InnerVolumeSpecName "kube-api-access-kth6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:14:34.934074 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.934044    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07be1ce0-5500-459d-8a1f-f92422a290e7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "07be1ce0-5500-459d-8a1f-f92422a290e7" (UID: "07be1ce0-5500-459d-8a1f-f92422a290e7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:14:34.939588 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.939563    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "07be1ce0-5500-459d-8a1f-f92422a290e7" (UID: "07be1ce0-5500-459d-8a1f-f92422a290e7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:14:34.949483 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:34.949462    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "07be1ce0-5500-459d-8a1f-f92422a290e7" (UID: "07be1ce0-5500-459d-8a1f-f92422a290e7"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:14:35.024636 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.024607    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/07be1ce0-5500-459d-8a1f-f92422a290e7-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:14:35.024636 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.024635    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:14:35.024773 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.024649    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-kserve-provision-location\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:14:35.024773 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.024661    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:14:35.024773 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.024676    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kth6s\" (UniqueName: \"kubernetes.io/projected/07be1ce0-5500-459d-8a1f-f92422a290e7-kube-api-access-kth6s\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:14:35.024773 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.024688    2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/07be1ce0-5500-459d-8a1f-f92422a290e7-cabundle-cert\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:14:35.024773 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.024699    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/07be1ce0-5500-459d-8a1f-f92422a290e7-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:14:35.714863 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.714832    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb" event={"ID":"07be1ce0-5500-459d-8a1f-f92422a290e7","Type":"ContainerDied","Data":"015dadd16262ac05a21b8df1d226eeeff35814a74d3c7e12fc04ff1b925fceae"}
Aug 05 18:14:35.715281 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.714914    2574 scope.go:117] "RemoveContainer" containerID="f3b4dadf6ccfa63f368869415e3365c6eb3ca596859f5c5060fc91dbaa524cdd"
Aug 05 18:14:35.715281 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.714845    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"
Aug 05 18:14:35.725857 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.725836    2574 scope.go:117] "RemoveContainer" containerID="bcbce8540fad269fb803ac91b10437baa89de942d397f210ac9d16027a929ccc"
Aug 05 18:14:35.737734 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.737712    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"]
Aug 05 18:14:35.741411 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.741390    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-d73d44f4/custom-route-timeout-pd-test-kserve-prefill-56c9496d6c-k4klb"]
Aug 05 18:14:35.853673 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:35.853648    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" path="/var/lib/kubelet/pods/07be1ce0-5500-459d-8a1f-f92422a290e7/volumes"
Aug 05 18:14:56.926825 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.926792    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"]
Aug 05 18:14:56.927330 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.927109    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="storage-initializer"
Aug 05 18:14:56.927330 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.927124    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="storage-initializer"
Aug 05 18:14:56.927330 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.927144    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main"
Aug 05 18:14:56.927330 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.927151    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main"
Aug 05 18:14:56.927330 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.927210    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="07be1ce0-5500-459d-8a1f-f92422a290e7" containerName="main"
Aug 05 18:14:56.930234 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.930216    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:56.933824 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.933794    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"router-with-refs-pd-test-kserve-self-signed-certs\""
Aug 05 18:14:56.933944 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.933830    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"default-dockercfg-4xrgk\""
Aug 05 18:14:56.936993 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.936842    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"openshift-service-ca.crt\""
Aug 05 18:14:56.936993 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.936892    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"kube-root-ca.crt\""
Aug 05 18:14:56.936993 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.936902    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"odh-kserve-custom-ca-bundle\""
Aug 05 18:14:56.937230 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.937004    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-test-llm-inference-service-b5dd93e6\"/\"seaweedfs-s3-creds\""
Aug 05 18:14:56.943091 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.943070    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"]
Aug 05 18:14:56.984153 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.984127    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:56.984262 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.984165    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-home\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:56.984262 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.984183    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d860376d-f42f-4771-9abf-9b8181c0ebae-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:56.984262 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.984198    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:56.984262 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.984217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-tmp-dir\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:56.984424 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.984266    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d860376d-f42f-4771-9abf-9b8181c0ebae-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:56.984424 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.984292    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:56.984424 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:56.984321    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb9bw\" (UniqueName: \"kubernetes.io/projected/d860376d-f42f-4771-9abf-9b8181c0ebae-kube-api-access-cb9bw\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085294 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085259    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085421 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085312    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-home\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085421 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085329    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d860376d-f42f-4771-9abf-9b8181c0ebae-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085421 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085421 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085368    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-tmp-dir\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085421 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085390    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d860376d-f42f-4771-9abf-9b8181c0ebae-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085421 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085407    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085752 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085439    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb9bw\" (UniqueName: \"kubernetes.io/projected/d860376d-f42f-4771-9abf-9b8181c0ebae-kube-api-access-cb9bw\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085752 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085674    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-model-cache\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085752 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085740    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-home\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085919 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085864    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-kserve-provision-location\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.085919 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.085897    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-tmp-dir\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.086364 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.086340    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d860376d-f42f-4771-9abf-9b8181c0ebae-cabundle-cert\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.087653 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.087634    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-dshm\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.087913 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.087896    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d860376d-f42f-4771-9abf-9b8181c0ebae-tls-certs\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.093772 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.093750    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb9bw\" (UniqueName: \"kubernetes.io/projected/d860376d-f42f-4771-9abf-9b8181c0ebae-kube-api-access-cb9bw\") pod \"router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") " pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.243074 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.242978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:14:57.376250 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.376161    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"]
Aug 05 18:14:57.380525 ip-10-0-141-246 kubenswrapper[2574]: W0805 18:14:57.380501    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd860376d_f42f_4771_9abf_9b8181c0ebae.slice/crio-68d4c1cf893ec26f91882269750060a7c034286a2cf561cd22fe30df930add1b WatchSource:0}: Error finding container 68d4c1cf893ec26f91882269750060a7c034286a2cf561cd22fe30df930add1b: Status 404 returned error can't find the container with id 68d4c1cf893ec26f91882269750060a7c034286a2cf561cd22fe30df930add1b
Aug 05 18:14:57.792988 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.792947    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" event={"ID":"d860376d-f42f-4771-9abf-9b8181c0ebae","Type":"ContainerStarted","Data":"9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc"}
Aug 05 18:14:57.793161 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:57.793034    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" event={"ID":"d860376d-f42f-4771-9abf-9b8181c0ebae","Type":"ContainerStarted","Data":"68d4c1cf893ec26f91882269750060a7c034286a2cf561cd22fe30df930add1b"}
Aug 05 18:14:59.801813 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:59.801727    2574 generic.go:356] "Generic (PLEG): container finished" podID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerID="9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc" exitCode=0
Aug 05 18:14:59.801813 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:14:59.801787    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" event={"ID":"d860376d-f42f-4771-9abf-9b8181c0ebae","Type":"ContainerDied","Data":"9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc"}
Aug 05 18:15:00.807869 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:00.807836    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" event={"ID":"d860376d-f42f-4771-9abf-9b8181c0ebae","Type":"ContainerStarted","Data":"b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab"}
Aug 05 18:15:00.830539 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:00.830475    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podStartSLOduration=2.784504298 podStartE2EDuration="4.830414896s" totalImagesPullingTime="0s" totalInitContainerRuntime="2.045910598s" isStatefulPod=false podCreationTimestamp="2026-08-05 18:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:15:00.828625917 +0000 UTC m=+3745.527881259" watchObservedRunningTime="2026-08-05 18:15:00.830414896 +0000 UTC m=+3745.529670242"
Aug 05 18:15:07.243246 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:07.243212    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:15:07.243656 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:07.243275    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:15:07.244762 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:07.244731    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:15:17.243671 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:17.243627    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:15:27.244199 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:27.244156    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:15:37.243628 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:37.243580    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:15:47.244223 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:47.244176    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:15:57.244165 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:15:57.244120    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:16:07.243877 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:16:07.243835    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:16:17.243693 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:16:17.243653    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:16:27.244307 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:16:27.244257    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:16:37.243719 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:16:37.243674    2574 prober.go:120] "Probe failed" probeType="Startup" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" probeResult="failure" output="Get \"https://10.134.0.32:8000/health\": dial tcp 10.134.0.32:8000: connect: connection refused"
Aug 05 18:16:47.253289 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:16:47.253251    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:16:47.260969 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:16:47.260945    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:17:11.064748 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:11.064675    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"]
Aug 05 18:17:26.086756 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.086701    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main" containerID="cri-o://b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab" gracePeriod=45
Aug 05 18:17:26.817361 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.817336    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:17:26.918580 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918557    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-home\") pod \"d860376d-f42f-4771-9abf-9b8181c0ebae\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") "
Aug 05 18:17:26.918728 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918599    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-model-cache\") pod \"d860376d-f42f-4771-9abf-9b8181c0ebae\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") "
Aug 05 18:17:26.918728 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918619    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-dshm\") pod \"d860376d-f42f-4771-9abf-9b8181c0ebae\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") "
Aug 05 18:17:26.918728 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918652    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d860376d-f42f-4771-9abf-9b8181c0ebae-tls-certs\") pod \"d860376d-f42f-4771-9abf-9b8181c0ebae\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") "
Aug 05 18:17:26.918728 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918667    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d860376d-f42f-4771-9abf-9b8181c0ebae-cabundle-cert\") pod \"d860376d-f42f-4771-9abf-9b8181c0ebae\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") "
Aug 05 18:17:26.918728 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918699    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-tmp-dir\") pod \"d860376d-f42f-4771-9abf-9b8181c0ebae\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") "
Aug 05 18:17:26.918728 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918726    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cb9bw\" (UniqueName: \"kubernetes.io/projected/d860376d-f42f-4771-9abf-9b8181c0ebae-kube-api-access-cb9bw\") pod \"d860376d-f42f-4771-9abf-9b8181c0ebae\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") "
Aug 05 18:17:26.919076 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918774    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-kserve-provision-location\") pod \"d860376d-f42f-4771-9abf-9b8181c0ebae\" (UID: \"d860376d-f42f-4771-9abf-9b8181c0ebae\") "
Aug 05 18:17:26.919076 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918902    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-model-cache" (OuterVolumeSpecName: "model-cache") pod "d860376d-f42f-4771-9abf-9b8181c0ebae" (UID: "d860376d-f42f-4771-9abf-9b8181c0ebae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:26.919076 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.918985    2574 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-model-cache\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:17:26.919237 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.919200    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d860376d-f42f-4771-9abf-9b8181c0ebae-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "d860376d-f42f-4771-9abf-9b8181c0ebae" (UID: "d860376d-f42f-4771-9abf-9b8181c0ebae"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Aug 05 18:17:26.919308 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.919246    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-home" (OuterVolumeSpecName: "home") pod "d860376d-f42f-4771-9abf-9b8181c0ebae" (UID: "d860376d-f42f-4771-9abf-9b8181c0ebae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:26.921116 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.921048    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-dshm" (OuterVolumeSpecName: "dshm") pod "d860376d-f42f-4771-9abf-9b8181c0ebae" (UID: "d860376d-f42f-4771-9abf-9b8181c0ebae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:26.921231 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.921112    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d860376d-f42f-4771-9abf-9b8181c0ebae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d860376d-f42f-4771-9abf-9b8181c0ebae" (UID: "d860376d-f42f-4771-9abf-9b8181c0ebae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Aug 05 18:17:26.921231 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.921146    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d860376d-f42f-4771-9abf-9b8181c0ebae-kube-api-access-cb9bw" (OuterVolumeSpecName: "kube-api-access-cb9bw") pod "d860376d-f42f-4771-9abf-9b8181c0ebae" (UID: "d860376d-f42f-4771-9abf-9b8181c0ebae"). InnerVolumeSpecName "kube-api-access-cb9bw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Aug 05 18:17:26.939432 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.939404    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d860376d-f42f-4771-9abf-9b8181c0ebae" (UID: "d860376d-f42f-4771-9abf-9b8181c0ebae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:26.950005 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:26.949981    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "d860376d-f42f-4771-9abf-9b8181c0ebae" (UID: "d860376d-f42f-4771-9abf-9b8181c0ebae"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Aug 05 18:17:27.020336 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.020314    2574 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d860376d-f42f-4771-9abf-9b8181c0ebae-tls-certs\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:17:27.020336 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.020333    2574 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/d860376d-f42f-4771-9abf-9b8181c0ebae-cabundle-cert\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:17:27.020473 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.020344    2574 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-tmp-dir\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:17:27.020473 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.020352    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb9bw\" (UniqueName: \"kubernetes.io/projected/d860376d-f42f-4771-9abf-9b8181c0ebae-kube-api-access-cb9bw\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:17:27.020473 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.020363    2574 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-kserve-provision-location\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:17:27.020473 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.020371    2574 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-home\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:17:27.020473 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.020381    2574 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d860376d-f42f-4771-9abf-9b8181c0ebae-dshm\") on node \"ip-10-0-141-246.ec2.internal\" DevicePath \"\""
Aug 05 18:17:27.390765 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.390675    2574 generic.go:356] "Generic (PLEG): container finished" podID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerID="b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab" exitCode=0
Aug 05 18:17:27.390765 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.390723    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"
Aug 05 18:17:27.390765 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.390747    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" event={"ID":"d860376d-f42f-4771-9abf-9b8181c0ebae","Type":"ContainerDied","Data":"b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab"}
Aug 05 18:17:27.391300 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.390790    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f" event={"ID":"d860376d-f42f-4771-9abf-9b8181c0ebae","Type":"ContainerDied","Data":"68d4c1cf893ec26f91882269750060a7c034286a2cf561cd22fe30df930add1b"}
Aug 05 18:17:27.391300 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.390811    2574 scope.go:117] "RemoveContainer" containerID="b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab"
Aug 05 18:17:27.399613 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.399592    2574 scope.go:117] "RemoveContainer" containerID="9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc"
Aug 05 18:17:27.406931 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.406916    2574 scope.go:117] "RemoveContainer" containerID="b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab"
Aug 05 18:17:27.407180 ip-10-0-141-246 kubenswrapper[2574]: E0805 18:17:27.407164    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab\": container with ID starting with b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab not found: ID does not exist" containerID="b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab"
Aug 05 18:17:27.407235 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.407188    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab"} err="failed to get container status \"b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab\": rpc error: code = NotFound desc = could not find container \"b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab\": container with ID starting with b45066f0ea66814e1e51803d13da8ef7988f729a128b7e2a5d7416968880b2ab not found: ID does not exist"
Aug 05 18:17:27.407235 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.407205    2574 scope.go:117] "RemoveContainer" containerID="9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc"
Aug 05 18:17:27.407416 ip-10-0-141-246 kubenswrapper[2574]: E0805 18:17:27.407398    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc\": container with ID starting with 9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc not found: ID does not exist" containerID="9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc"
Aug 05 18:17:27.407462 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.407422    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc"} err="failed to get container status \"9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc\": rpc error: code = NotFound desc = could not find container \"9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc\": container with ID starting with 9256b838cebd3a953fa279aeafaadb7268cf5c03f9733583962ca5f9a3ee23fc not found: ID does not exist"
Aug 05 18:17:27.412138 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.412117    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"]
Aug 05 18:17:27.415739 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.415718    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-test-llm-inference-service-b5dd93e6/router-with-refs-pd-test-kserve-prefill-6c6c444d55-5592f"]
Aug 05 18:17:27.854160 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:17:27.854127    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" path="/var/lib/kubelet/pods/d860376d-f42f-4771-9abf-9b8181c0ebae/volumes"
Aug 05 18:20:28.188442 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:28.188380    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-75xjq_edae28b3-e05b-4992-91a1-2cc8c9645f36/discovery/0.log"
Aug 05 18:20:29.132604 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:29.132575    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-75xjq_edae28b3-e05b-4992-91a1-2cc8c9645f36/discovery/0.log"
Aug 05 18:20:30.386197 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:30.386171    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-nphpf_a81ccf7e-3e7f-476e-8615-2e512c116572/kuadrant-console-plugin/0.log"
Aug 05 18:20:37.279343 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:37.279308    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-8gjld_de68f860-081a-403c-8a3d-6299868bc425/global-pull-secret-syncer/0.log"
Aug 05 18:20:37.389028 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:37.388999    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-85lrg_b8083658-9899-4a1b-b99d-b97e1ad222b4/konnectivity-agent/0.log"
Aug 05 18:20:37.508861 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:37.508833    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-246.ec2.internal_0ea1cdff287318a9cf0bdf169156a32c/haproxy/0.log"
Aug 05 18:20:42.908608 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:42.908576    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-nphpf_a81ccf7e-3e7f-476e-8615-2e512c116572/kuadrant-console-plugin/0.log"
Aug 05 18:20:44.248256 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.248224    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e36d99-e603-46ab-bcf5-6ff13f01d33d/alertmanager/0.log"
Aug 05 18:20:44.272688 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.272665    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e36d99-e603-46ab-bcf5-6ff13f01d33d/config-reloader/0.log"
Aug 05 18:20:44.295751 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.295726    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e36d99-e603-46ab-bcf5-6ff13f01d33d/kube-rbac-proxy-web/0.log"
Aug 05 18:20:44.321366 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.321346    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e36d99-e603-46ab-bcf5-6ff13f01d33d/kube-rbac-proxy/0.log"
Aug 05 18:20:44.341856 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.341831    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e36d99-e603-46ab-bcf5-6ff13f01d33d/kube-rbac-proxy-metric/0.log"
Aug 05 18:20:44.361341 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.361321    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e36d99-e603-46ab-bcf5-6ff13f01d33d/prom-label-proxy/0.log"
Aug 05 18:20:44.382757 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.382737    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_13e36d99-e603-46ab-bcf5-6ff13f01d33d/init-config-reloader/0.log"
Aug 05 18:20:44.587354 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.587291    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-6dffbbd67-8mgrr_4e868dc0-89f8-47a4-8ffc-ac39f1b08daf/monitoring-plugin/0.log"
Aug 05 18:20:44.845515 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.845440    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zptlm_e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb/node-exporter/0.log"
Aug 05 18:20:44.890788 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.890756    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zptlm_e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb/kube-rbac-proxy/0.log"
Aug 05 18:20:44.947923 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:44.947903    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zptlm_e3cbb0d3-c038-4b4d-b5bf-8c32e32ff5cb/init-textfile/0.log"
Aug 05 18:20:45.302842 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:45.302804    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b98b597b4-dls9w_2e485294-1501-4a6d-96ca-809e78b097f5/telemeter-client/0.log"
Aug 05 18:20:45.322633 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:45.322608    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b98b597b4-dls9w_2e485294-1501-4a6d-96ca-809e78b097f5/reload/0.log"
Aug 05 18:20:45.357792 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:45.357770    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5b98b597b4-dls9w_2e485294-1501-4a6d-96ca-809e78b097f5/kube-rbac-proxy/0.log"
Aug 05 18:20:47.373673 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.373636    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"]
Aug 05 18:20:47.374169 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.374050    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="storage-initializer"
Aug 05 18:20:47.374169 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.374066    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="storage-initializer"
Aug 05 18:20:47.374169 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.374082    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main"
Aug 05 18:20:47.374169 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.374090    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main"
Aug 05 18:20:47.374380 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.374181    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d860376d-f42f-4771-9abf-9b8181c0ebae" containerName="main"
Aug 05 18:20:47.377289 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.377266    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.379269 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.379253    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zgx78\"/\"openshift-service-ca.crt\""
Aug 05 18:20:47.379350 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.379269    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zgx78\"/\"kube-root-ca.crt\""
Aug 05 18:20:47.379981 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.379955    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zgx78\"/\"default-dockercfg-cb4lc\""
Aug 05 18:20:47.383339 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.383316    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"]
Aug 05 18:20:47.519609 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.519580    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-proc\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.519792 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.519625    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-lib-modules\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.519792 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.519681    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-podres\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.519792 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.519723    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-sys\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.519792 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.519784    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzjxv\" (UniqueName: \"kubernetes.io/projected/e4a0170c-c05c-42ba-adbb-bd34077d2557-kube-api-access-nzjxv\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.620116 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.620091    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzjxv\" (UniqueName: \"kubernetes.io/projected/e4a0170c-c05c-42ba-adbb-bd34077d2557-kube-api-access-nzjxv\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.620238 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.620144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-proc\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.620238 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.620201    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-proc\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.620356 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.620256    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-lib-modules\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.620356 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.620296    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-podres\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.620356 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.620327    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-sys\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.620505 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.620424    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-sys\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.620505 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.620446    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-lib-modules\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.620505 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.620446    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e4a0170c-c05c-42ba-adbb-bd34077d2557-podres\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.627284 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.627234    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzjxv\" (UniqueName: \"kubernetes.io/projected/e4a0170c-c05c-42ba-adbb-bd34077d2557-kube-api-access-nzjxv\") pod \"perf-node-gather-daemonset-l47pz\" (UID: \"e4a0170c-c05c-42ba-adbb-bd34077d2557\") " pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.686726 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.686706    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:47.811931 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:47.811900    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"]
Aug 05 18:20:47.814425 ip-10-0-141-246 kubenswrapper[2574]: W0805 18:20:47.814388    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode4a0170c_c05c_42ba_adbb_bd34077d2557.slice/crio-7d79de2b9a94046b70c9cc32175a3d2f3f2da51f43626739deb179c2af0275ed WatchSource:0}: Error finding container 7d79de2b9a94046b70c9cc32175a3d2f3f2da51f43626739deb179c2af0275ed: Status 404 returned error can't find the container with id 7d79de2b9a94046b70c9cc32175a3d2f3f2da51f43626739deb179c2af0275ed
Aug 05 18:20:48.116318 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:48.116281    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz" event={"ID":"e4a0170c-c05c-42ba-adbb-bd34077d2557","Type":"ContainerStarted","Data":"23edcbf7b6aedece0c6d21790276b3559d5f26314123ea27948b08e51edf9560"}
Aug 05 18:20:48.116478 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:48.116345    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz" event={"ID":"e4a0170c-c05c-42ba-adbb-bd34077d2557","Type":"ContainerStarted","Data":"7d79de2b9a94046b70c9cc32175a3d2f3f2da51f43626739deb179c2af0275ed"}
Aug 05 18:20:48.116478 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:48.116392    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:48.130795 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:48.130750    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz" podStartSLOduration=1.130738399 podStartE2EDuration="1.130738399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-08-05 18:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-08-05 18:20:48.130446804 +0000 UTC m=+4092.829702146" watchObservedRunningTime="2026-08-05 18:20:48.130738399 +0000 UTC m=+4092.829993741"
Aug 05 18:20:49.184917 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:49.184888    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9p8vp_5a71b652-b075-4a6e-be90-bc33a101e34b/dns/0.log"
Aug 05 18:20:49.203683 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:49.203655    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9p8vp_5a71b652-b075-4a6e-be90-bc33a101e34b/kube-rbac-proxy/0.log"
Aug 05 18:20:49.286353 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:49.286325    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-llnmn_cc0aaa1f-a8d6-41c5-8f22-c5bde34131a4/dns-node-resolver/0.log"
Aug 05 18:20:49.868355 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:49.868326    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s5rqv_975a4dd9-7be5-40da-b2ce-ab8162b1d289/node-ca/0.log"
Aug 05 18:20:50.977235 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:50.977204    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-75xjq_edae28b3-e05b-4992-91a1-2cc8c9645f36/discovery/0.log"
Aug 05 18:20:51.581884 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:51.581853    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n46df_3968697f-5e9f-4e47-8514-3afa61d501f0/serve-healthcheck-canary/0.log"
Aug 05 18:20:52.193890 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:52.193860    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shjh5_731c2e93-915e-4456-8eef-c862ced23934/kube-rbac-proxy/0.log"
Aug 05 18:20:52.212541 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:52.212517    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shjh5_731c2e93-915e-4456-8eef-c862ced23934/exporter/0.log"
Aug 05 18:20:52.233476 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:52.233454    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-shjh5_731c2e93-915e-4456-8eef-c862ced23934/extractor/0.log"
Aug 05 18:20:54.129654 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:54.129628    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zgx78/perf-node-gather-daemonset-l47pz"
Aug 05 18:20:55.249207 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:55.249176    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-77c595c7bb-7cb8n_88467a88-d0f1-4fb6-816c-5b8d2aacf890/manager/0.log"
Aug 05 18:20:57.298557 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:57.298533    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-75xjq_edae28b3-e05b-4992-91a1-2cc8c9645f36/discovery/0.log"
Aug 05 18:20:58.756682 ip-10-0-141-246 kubenswrapper[2574]: I0805 18:20:58.756640    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-75xjq_edae28b3-e05b-4992-91a1-2cc8c9645f36/discovery/0.log"